searchstax / searchstax-cloud-connector-for-sitecore Goto Github PK

The aliases and collections used for the XDB rebuild collections are not prefixed by default. This means that you cannot have multiple sites running on the same instance without alterations.

Regular PowerShell must be used for Azure Apps.

It could either be

Two options being presented in the Yaml module and pros/cons listed in the plugin documentation
Option a) Use standard solr configs (-1 autoSoftCommit, 15000ms autoCommit)
Option b) Use SearchStax recommended configs (120000ms autoSoftCommit, 300000ms autoCommit)

Or simply using SearchStax recommended configs by default. This would provide a more sensible onboarding experience and reduce post-onboarding support tickets for performance issues and backup failures.

SearchStax recommendations are here for reference - https://www.searchstax.com/blog/5-ways-to-optimize-your-solr-search-performance-for-sitecore/

This plugin assumes too much about how a Sitecore site might be set up. For instance, if my prefix is mylittlepony and root is C:\inetpub\wwwroot it assumes the connection strings are located in C:\inetpub\wwwroot\mylittlepony.xconnect \App_Config\ConnectionStrings.config. This is quite often not the case as many of the Sitecore SIF installation steps are customizable.

Finding account names, deployment IDs and instructing users to user their SearchStax passwords (Or 2FA) rather than deployment passwords will help developers become more self sufficient with the connector

If clients have MFA enabled, plugin should ask for the TFA token. Currently it does not, and the authentication fails and clients are not able to use the plugin

As part of #2 , I have discovered that the aliases are required when using SwitchOnRebuildSolrCloudSearchIndex. The preference is to set the setting ContentSearch.Solr.EnforceAliasCreation to false in Sitecore, so that everything that is required from Solr is created using this plugin.

This is an un-used feature which requires additional maintainability for the connector

Criteria:
When the user specifies Configuration Mode, COMMERCE should not be an accepted option.

ZD Tkt - #5119

Creating XDB Aliases ...
xdb_internal
Invoke-WebRequest: C:\pkgs\searchstax-sitecore-plugin-1.15\searchstax-sitecore-plugin-1.15\src\searchstax-sitecore-xconnect.ps1:58
Line |
58 | Invoke-WebRequest -Uri $url
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~
| { "responseHeader":{ "status":400, "QTime":28}, "Operation createalias caused
| exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Can't create
| collection alias for collections='xdb_internal', 'xdb_internal' is not an existing collection or
| alias", "exception":{ "msg":"Can't create collection alias for collections='xdb_internal',
| 'xdb_internal' is not an existing collection or alias", "rspCode":400}, "error":{
| "metadata":[ "error-class","org.apache.solr.common.SolrException",
| "root-error-class","org.apache.solr.common.SolrException"], "msg":"Can't create collection alias
| for collections='xdb_internal', 'xdb_internal' is not an existing collection or alias",
| "code":400}}

xdb_rebuild_internal
Invoke-WebRequest: C:\pkgs\searchstax-sitecore-plugin-1.15\searchstax-sitecore-plugin-1.15\src\searchstax-sitecore-xconnect.ps1:58
Line |
58 | Invoke-WebRequest -Uri $url
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~
| { "responseHeader":{ "status":400, "QTime":10}, "Operation createalias caused
| exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Can't create
| collection alias for collections='xdb_rebuild_internal', 'xdb_rebuild_internal' is not an existing
| collection or alias", "exception":{ "msg":"Can't create collection alias for
| collections='xdb_rebuild_internal', 'xdb_rebuild_internal' is not an existing collection or alias",
| "rspCode":400}, "error":{ "metadata":[
| "error-class","org.apache.solr.common.SolrException",
| "root-error-class","org.apache.solr.common.SolrException"], "msg":"Can't create collection alias
| for collections='xdb_rebuild_internal', 'xdb_rebuild_internal' is not an existing collection or
| alias", "code":400}}

Updating XConnect ConnectionStrings in 'D:\sitecore[sitecore.cm](http://sitecore.cm/)\sitecore.xconnect\App_Config\ConnectionStrings.config' file
Write-Error: C:\pkgs\searchstax-sitecore-plugin-1.15\searchstax-sitecore-plugin-1.15\src\searchstax-sitecore-xconnect.ps1:74
Line |
74 | Update-XML $path $xpath $attributeKey $attributeValue
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Could not find D:\sitecore[sitecore.cm](http://sitecore.cm/)\sitecore.xconnect\App_Config\ConnectionStrings.config File

Updating XConnect ConnectionStrings in 'D:\sitecore[sitecore.cm](http://sitecore.cm/)\sitecore.xconnect\App_Data\jobs\continuous\IndexWorker\App_Config\ConnectionStrings.config' file
Write-Error: C:\pkgs\searchstax-sitecore-plugin-1.15\searchstax-sitecore-plugin-1.15\src\searchstax-sitecore-xconnect.ps1:74
Line |
74 | Update-XML $path $xpath $attributeKey $attributeValue
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Could not find
| D:\sitecore[sitecore.cm](http://sitecore.cm/)\sitecore.xconnect\App_Data\jobs\continuous\IndexWorker\App_Config\ConnectionStrings.config File

Get-Content: C:\pkgs\searchstax-sitecore-plugin-1.15\searchstax-sitecore-plugin-1.15\src\searchstax-sitecore-xconnect.ps1:86
Line |
86 | $json = Get-Content -Raw -Path $path
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Cannot find path 'D:\sitecore[sitecore.cm](http://sitecore.cm/)\sitecore.xconnect\App_Data\solrcommands\schema.json' because
| it does not exist.

https://ss856956-knf6sz36-us-west-2-aws.searchstax.com/solr/
Updating XDB Schema ...
xdb_internal
Invoke-RestMethod: C:\pkgs\searchstax-sitecore-plugin-1.15\searchstax-sitecore-plugin-1.15\src\searchstax-sitecore-xconnect.ps1:102
Line |
102 | Invoke-RestMethod -Uri $url -ContentType 'application/jso …
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Error 404 Not Found HTTP ERROR 404 Problem accessing /solr/xdb_internal/schema. Reason: Not
| Found

xdb_rebuild_internal
Invoke-RestMethod: C:\pkgs\searchstax-sitecore-plugin-1.15\searchstax-sitecore-plugin-1.15\src\searchstax-sitecore-xconnect.ps1:102
Line |
102 | Invoke-RestMethod -Uri $url -ContentType 'application/jso …
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Error 404 Not Found HTTP ERROR 404 Problem accessing /solr/xdb_rebuild_internal/schema. Reason:
| Not Found

Restarting IIS
NOTE: If you have UAC enabled, then this step might fail with 'Access Denied' error.
Please either disable UAC, or restart IIS manually if the error occurs.

Keep xConnect schema file in the same zip directory as the same way other Solr configs are.

Add support for XDB Indexes

I propose the scripts to be re-organised, so that it no longer needs an external yaml file, rather all the parameters can be passed in via script arguments instead. This will allow a user to run this on their developer PC, as well as on a CI / CD tool like Azure DevOps. Here is how I will break it down.

1. Do not zip the Solr config in the repo, rather fully expand them so that all users can see all the content.
2. Allow independent naming of Solr Configuration that will be uploaded, and the Index Cores collection prefixes.
3. Add a Powershell module to zip the Solr configurations temporarily when Posting to the Solr API
4. Add more index collection support for XDB, SXA and Sitecore Commerce.
5. Provide documentation inside Powershell scripts, and Readme file, so that users have a good idea on how to use the scripts and add their own custom index collections.
6. Add script to add and remove IP filtering

I will progressively work through this.

With the release of Experience Edge and its associated connector, an additional Solr collection is required for this connector to function.

https://doc.sitecore.com/en/developers/101/developer-tools/the-experience-edge-connector.html

Requirements:

• Powershell interface
• Support for SearchStax 2FA/MFA authentication if it's enabled for the SearchStax user account
• All configuration files created include 300000 maxTime for autoCommit and 120000 maxTime for autoSoftCommit for all Solr config.xml files
• Easier Maintainability
  • Keep xConnect schema file in the same zip directory as the same way other Solr configs are. (#48)
  • Remove On-Premise configuration changes to Sitecore and assume it is Azure PaaS (#47)
  • Remove Sitecore Commerce Support (#46)

User should be able to specify:

• Sitecore version (This exists already)

  • Compatible version of Sitecore configs is uploaded to target deployment

• Configuration Mode (XM, XP, XCONNECT) (This exists already)

  • Default Sitecore indexes are created for the configuration mode selected

• If SXA is required (This exists already)

  • If yes, then prefix_sxa_web_index and prefix_sxa_master_index are created

• If XConnect is required (This exists already)

  • If yes, Rebuild index for xDB and aliases are configured for SwitchOnRebuild support

• If SwitchOnRebuild is enabled for Sitecore indexes (New)

  • If yes, then Rebuild indexes and corresponding mainAliases and rebuildAliases are created for each Index by default using the values below provided by the user
  • The rebuildSuffix (README File can guide them on whether "_" preceding rebuildSuffix is provided by the user or added automatically)
  • The mainAliasSuffix
  • The rebuildAliasSuffix

• Custom Index name (New)

  • User can pass an list of custom index names
  • User can also specify for each custom index, if it will have SwitchOnRebuild
    - If yes, then use the values provided in the SwitchOnRebuild section above for rebuildSuffix, mainAliasSuffix and rebuildAliasSuffix

Prospect ran Sitecore Plugin, got an error saying that it could not overwrite an existing configset. (Our Zookeeper API "create" will not overwrite a configset.) Prospect was running the plugin in XCONNECT mode, and an existing xDB configset broke it.

Suggest either (1) fix the Zookeeper API "create" method to overwrite like zkcli does, or
(2) fix Plugin to delete existing configsets before writing new ones. BDC

• Remove PathtoRoot (pathToWWWRoot) field from YAML file since this change will not be made in Sitecore.
• Remove isAzurePaaS field from YAML file and always assume TRUE for this

Currently this can be run in On-Premise or Remotely modes. New implementation will only be for Remotely mode.

Criteria:
If a user runs this script with on-premise Sitecore, the changes to Sitecore configuration files should not happen.

As per https://doc.sitecore.com/en/developers/100/platform-administration-and-architecture/walkthrough--setting-up-solrcloud.html,

"If you only access nodes of SolrCloud within a private network, set the ContentSearch.IndexingManager.DisplayShortStatistic setting to true. This eliminates direct requests to the nodes used to display additional index statistics in the Indexing Manager Wizard."

Enhance the script to set this to true for SItecore versions 10.0 and above

For easier maintainability of Sitecore version support, download the default_ config from an existing Solr deployment, modify that and re-upload it when creating new configs with connector