Sharding Custom Entity in Sitecore Commerce 10

Sharding is horizontal partitioning of data in database. It is the process of breaking up large tables into smaller chunks.

Storing rows of the same table in multiple database nodes

In this blog post will see how to split the Commerce Entities table having same structure but store custom entity data in a separate table that helps to split the load that a CommerceEntities table might take if the horizontal partition is not done.

Why partioning of tables is required?

Sitecore Commerce entity data are store in following tables-

  • CommerceEntities
  • CommerceEntity
  • CommerceLists

Any custom entity been created without sharding will store data by default in tables mentioned above.

What if that data increases, there might be a performance hit once the data start expanding over months and years.

Also it wont be good idea to put the multiple custom entity data into a single table. As this might give a performance hit whilst indexing table. So, if you know data might increase over the time it is better to have it saved in a separate table as it can be a boon to high-volume data.

Partitioning data using sharding policy

I assume you know how extend Sitecore Commerce entities. Consider we have a “Organization” entity. Business Tools helps in capturing details of Organization i.e. CRUD operations. When the entity is been saved it has to be saved in different table.

This driven by the sharding policies in Commerce.

Follow these steps to enable sharding of custom entity-

Sharding Policies

The Commerce Engine implements database sharding for Commerce entity and list tables, and provides 2 types of sharding policies. One is for the operation against Commerce entities i.e. EntityShardingPolicy, and other on the Commerce lists i.e. ListShardingPolicy.

Configuration for sharding policy is kept in PlugIn.SQL.Sharding.PolicySet-1.0.0.json file and can be found in data\Environments folder of the Authoring and Shops instance.

As per Sitecore documentation sharding policies has expressions and multiple expression values can be configured based on this the table of the entity is identified to read and perform write operations. This is a bit contrary statement as the table name defined in policies are passed to the stored procedure based on this the data in table is written and read.

Below sharding policy mentions 2 tables-

OrganizationsLists for managing and reading lists of Organizations

OrganizationsEntities for managing and reading Organization entities

{
"$type": "Sitecore.Commerce.Plugin.SQL.ListShardingPolicy, Sitecore.Commerce.Plugin.SQL",
"Expressions": {
"$type": "System.Collections.Generic.List1[[System.String, mscorlib]], mscorlib",
"$values":
[
"^List-Organization.*?$"
]
},
"TableName": "OrganizationsLists"
},
{ 
"$type": "Sitecore.Commerce.Plugin.SQL.EntityShardingPolicy, Sitecore.Commerce.Plugin.SQL",
"Expressions": {
"$type": "System.Collections.Generic.List1[[System.String, mscorlib]], mscorlib",
"$values":
[
"^Entity-Organization.?$",
"^Organization-.?$"
]
},
"TableName": "OrganizationsEntities"
}

Database

To save data in different tables create Entities, Entity and Lists table prefixed with entity name in SitecoreCommerce_SharedEnvironments database

  1. Right click CommerceEntities table select Script Table as option, Create to and then New Query Editor Window. Create script for CommerceEntities table will be generated.
  2. Change the name of table to e.g.:- OrganizationsEntities. Also change the table name to set Default value to EntityVersion and Published fields
  3. Pasrse and check if you are creating a table in correct Database
  4. Execute the script. New table will be created.

Follow same for CommerceEntity table to create OrganizationsEntity and CommerceLists to create OrganizationsLists

So there are 3 tables created so far-

  • OrganizationsEntities
  • OrganizationsEntity
  • OrganizationsLists

Once you have your plugin to perform CRUD operations on Organizations entity you should be able to see the data been inserted in OrganizationEntities, OrganizationLists and OrganizationsEntity table instead of CommerceEntities and there related tables.

Sitecore Commerce 10- Setup Development Environment for Business Tools

Business Tools is extensible using pluggable framework and can extend a UI using Entity Views. Although Business Tools offers a rich set of controls you might want to create your own custom control for the best business experience.

For this you need to first setup the development environment for business tools. Once the environment is setup you should be ready to develop custom control/customize the business tools.

Prerequisites

  1. Instance of Commerce Engine deployed in development environment
  2. Install Node.js Javascript runtime
  3. Install Angular CLI tool – npm install -g @angular/cli

NPM configuration to have NPM Packages from Sitecore public feed

Sitecore BizFx SDK relies on NPM packages available on the Sitecore official public feed for NPM packages.

Open Poswershell as Administrator

Execute these 2 commands in powershell

npm config set @speak:registry=https://sitecore.myget.org/F/sc-npm-packages/npm/

npm config set @sitecore:registry=https://sitecore.myget.org/F/sc-npm-packages/npm/

This will add following line to–

C:\Users\[your user]\.npmrc

[Optional] – you may check if this lines are added

@speak:registry=https://sitecore.myget.org/F/sc-npm-packages/npm/
@sitecore:registry=https://sitecore.myget.org/F/sc-npm-packages/npm/

Setup and Install SPEAK and BizFx packages for development solution

Copy Sitecore.BizFX.SDK.4.0.8 folder to your development folder and extract the SDK zip file to folder e.g. c:\BizFXDevelopment\SitecoreBizFx

Copy below files to the folder SDK was extracted. You should find this files from the Sitecore XC release package.

  1. speak-icon-fonts-1.1.0.tgz
  2. speak-ng-bcl-2.0.0-r00116.tgz
  3. speak-styling-1.0.0-r00110.tgz

Execute the following commands where the above files were copied

​​​​​​​npm install speak-icon-fonts-1.1.0.tgz
​​​​​​​npm install speak-ng-bcl-2.0.0-r00116.tgz
npm install speak-styling-1.0.0-r00110.tgz
npm install @sitecore/bizfx

Run npm install. This should install required npm modules and add a folder node_modules

npm install

Setup the business tools config.json with your deployment configuration

Once the npm installed successfully open config.json file located in src\assets folder

Update the config to the same as the BizFx site instance except for BizFxUri. Note BizFxUri points to http in below config

{
  "EnvironmentName": "HabitatAuthoring",
  "EngineUri": "https://localhost:5000",
  "IdentityServerUri": "https://xp10.IdentityServer",
  "BizFxUri": "http://localhost:4200",
  "Language": "en",
  "ContentLanguage": "en",
  "Currency": "USD",
  "ShopName": "CommerceEngineDefaultStorefront",
  "LanguageCookieName": "selectedLanguage",
  "ContentLanguageCookieName": "selectedContentLanguage",
  "EnvironmentCookieName": "selectedEnvironment",
  "AutoCompleteTimeout_ms": 300,
  "AccessTokenUpdateInterval_ms": 300000
}

Run the development environment

Important!

Stop the SitecoreBizFx site as the site listens to 4200 port. Next step will help listen the site from the extracted SDK folder.

Execute following Powershell command –

ng server

Open browser on http://localhost:4200/ this should ask to enter the Sitecore client credentials, once provided it will throw an error

This site can’t provide a secure connection

The reason this error occurs the identity server is not configure to server BizFx site on http

Update the Sitecore Identity Server Configuration

Open the Sitecore.Commerce.Identity ServiceHost.xml from the installed Identity Server instance \wwwroot\Config\production

Add http://localhost:4200 to AllowedCorsOriginGroup1

<AllowedCorsOrigins>
<AllowedCorsOriginsGroup1>http://localhost:4200</AllowedCorsOriginsGroup1>
</AllowedCorsOrigins>

Update Commerce Engine configuration

  • Open config.json from wwwroot folder in CommerceAuthoring site
  • Since the config is changed need to bootstrap so the changes are applied to authoring site
  • Restart IIS. Optionally you may just restart Commerce Authoring site

Run Business tool from development environment

Open browser on http://localhost:4200/

Business tools running on http and in developer mode.

References –

step-by-step instructions on how to setup and compile the Business Tools (BizFX) application using the ​Sitecore.BizFX.SDK

Stay tuned next blog will walk through on how to create a new custom control/component in Business Tools

This image has an empty alt attribute; its file name is image-43.png

Setup development environment for Sitecore Commerce Engine 10

Sitecore Experience Commerce 10 has come up with great new features like Dynamic Bundles, Free gift with Purchase promotion and a sample Sitecore DAM to Commerce connector.

Before you start looking into this, it is important to setup the development environment to debug and test the changes you are making to engine.

Main changes I could see compared to previous versions are integration with Content Hub and Configuring the Commerce Engine using environment variables which not only helps for on-premise installation of Commerce Instance but also helps setup the Docker technology where XC solution is running in containers.

In this post I will walk you through on how to setup the development environment. This post assumes you have Sitecore Commerce Engine along with Visual Studio 2019 installed on developer workstation. If Commerce not installed no worries see this post on how to install Sitecore XC 10 step-by-step.

Step-by-step install Sitecore Commerce (XC) 10

For previous version of XC you may follow this blog post

Step-by-step – Setup development environment for Sitecore Commerce 9.3 Engine

Step 1- Extract Commerce Engine SDK

  • Copy the downloaded SDK Sitecore.Commerce.Engine.SDK.6.0.130.zip on your development folder. e.g: c:\development. Note– there is an update on 19th August where the external dependencies are removed. Download the package again if you have a version before this date.
  • If not available you may download Packages for On Premise WDP 2020.08-6.0.238. Login before you download the file.
  • Extract the commerce package and then extract Sitecore.Commerce.Engine.SDK.6.0.130.zip in your development folder

Step 2 – Setup Visual Studio Solution

  • Whilst opening solution login from slpartners.myget.org will be prompted
  • Create an account on https://slpartner.myget.org/ and login here. You may unload Plugin.Sample.ContentHub project if you dont want to integrate ContentHub and the login should not require. Also note myget account has a trial for 14 days.
  • Build the Solution. It should restore the package and build successfully.
  • (optional)Rename the Solution name. In this case I have renamed to Retail.Commerce
  • (Optional) Create Foundation and Feature projects. Build the solution again.

Step 3- Important – Commerce Engine configuration

Sitecore.Commerce.Engine project should have a config.json file in wwwroot folder. Open this file you will see the placeholders that needs to be filled in.

Instead updating config file, you should update the launchSettings.json and the placeholders in config.json will be updated on launch on commerce engine.

Similarly Global.json you can find this in wwwroot/bootstrap folder of your Sitecore.Commerce.Engine project. Again this file has the Placeholders that will be populated from launchSettings.json,

You need to update mainly following variables in launchSettings.json file for both config and global json. There are other variables apart from listed below, you may need to update those based on your site instance name etc.-

  1. COMMERCEENGINE_Caching__Redis__Options__Configuration
  2. COMMERCEENGINE_GlobalDatabaseServer
  3. COMMERCEENGINE_GlobalDatabaseUserName
  4. COMMERCEENGINE_GlobalDatabasePassword
  5. COMMERCEENGINE_SharedDatabaseServer
  6. COMMERCEENGINE_SharedDatabaseUserName
  7. COMMERCEENGINE_SharedDatabasePassword
  8. COMMERCEENGINE_AppSettings__SitecoreIdentityServerUrl
  9. COMMERCEENGINE_EngineAuthoringUrl
  10. COMMERCEENGINE_EngineShopsUrl
  11. COMMERCEENGINE_EngineMinionsUrl
  12. COMMERCEENGINE_EngineHealthCheckUrl
  13. COMMERCEENGINE_AppSettings__AllowedOrigins

Step 4 – Generate Development Certificate

Generate development certificate using script “New-DevelopmentCertificate”, so the localhost runs on SSL(https)

  1. Create a folder named “dev” in the root directory of SDK
  2. Create a folder named “Sitecore.Commerce.Engine_Dev” under “dev” folder
  3. Create a folder named “wwwroot” under “Sitecore.Commerce.Engine_Dev” folder
  4. Open powershell script and navigate to scripts folder.
  5. Change the Path($certificateOutputDirectory) if required. Certificate should be copied to \src\Project\Engine\code\wwwroot
  6. Execute New-DevelopmentCertificate script. This script should be available in script folder in SDK folder.

Step 5 – Update EngineUri in BizFx Site

Step 6- Run the Commerce Engine from Visual Studio

  • Set the Sitecore.Commerce.Engine project as Startup Project
  • Change the emulator to Engine
  • Stop the CommerceAuthoring_SC site hosted in IIS
  • Run the solution

Hope there should be nothing that should block to run the Business Tools requesting a call to Engine running from Visual Studio

Note: some places you may have to restart IIS also clear the browser cache before you start checking Business Tools is highly recommended.

Hope this post helps you setting your XC 10 development environment.

ISSUES

Request origin https://bizfx.sc.com does not have permission to access the resource

Resolution- Follow Step 5

Options to Install Sitecore (XP) and Commerce (XC) 10

If you are looking for upgrade to Sitecore 10, below are the various options you are able to install Sitecore 10.

Sitecore XP 10

Sitecore Installation Assistant (SIA)

Sitecore Installation Assistant helps guides you through the Sitecore XP Developer Workstation installation. Use this option to review system requirements, install prerequisites and complete the entire installation process. With Sitecore 10 you have a option to also install SXA with SIA.

To install Sitecore 10 using SIA follow this post- Step-by-step how to install Sitecore 10 using SIA

Sitecore Installation Framework On-Premises (SIF)

Sitecore Install Framework (SIF) is a Microsoft PowerShell module that supports local and remote installations of Sitecore Experience Platform.

SIF deploys Web Deploy Packages (WDP) by passing parameters to SIF configuration through a Microsoft PowerShell module and is fully extensible.
The Sitecore Experience Platform is designed to be secure-by-default. For developer environments all the required self-signed certificates are created automatically if you do not provide any.
In a production environment, you can provide your own certificates In a non-production environment, you can choose to have the module generate the certificates for you.

You must set up SIF before you can install Sitecore Experience Platform

To install Sitecore XP 10 using SIF follow this post – Step-by-step install Sitecore XP 10 using Sitecore Installation Framework (SIF)

Sitecore Containers

Sitecore Containers support rapid deployment and more efficient solution and team onboarding with modern Docker and Kubernetes technology.

Sitecore Experience Platform 10.0.0 uses Docker Compose as the container
orchestrator on developer workstations. Docker Compose is a simple containerdeployment tool that is bundled with Docker for Windows. Sitecore container images can be deployed with other tools but we recommend that you use Docker Compose to deploy the containers that form the Sitecore Experience Platform.

To install Sitecore XP 10 using Sitecore Containers with Docker Compose- Step-by-step install Sitecore XP 10 to developer workstation using Sitecore Containers with Docker Compose

Sitecore XC 10

Sitecore Installation Framework (SIF)

Sitecore Install Framework (SIF) is a Microsoft PowerShell module that supports local and remote installations of Sitecore Experience Platform.

The SIF.Sitecore.Commerce package contains Sitecore Installation Framework scripts and Web Deployment Packages (WDP).

To install Step-by-step install Sitecore Commerce (XC) 10

Install Sitecore XP 10 using Sitecore Installation Assistant(SIA) in easy steps to development machine

Installing Sitecore 10 with Sitecore Installation Assistant (SIA) is quick and easy. Only manual prerequisite is to install SQL Server 2017

Please see through screens to install Sitecore 10-

Step 1- Install SQL Server 2017 and SQL Server Management Studio (SSMS)

Step 2- Download Sitecore 10 Graphical setup package for XP Single

Step 3- Extract the zip and start setup

Step 4-

Step 5- Install prerequisite. This should install SIF and Windows Server prerequisites including IIS and modules etc.

Step 6 – Once prerequisites are installed successfully yo may have to restart machine. Install Solr search service. Sitecore 10 needs SOLR 8.4

Step 7- After SOLR is installed enter the Sitecore settings along with SQL server details and SOLR details

Step 8 – Optionally select Sitecore modules (SXA)

That’s it and everything is managed by SIA to sucessfuly install Sitecore 10 on machine

Enable Catalog search in business tool for Sitecore 9.1 Commerce environments

Sitecore Commerce indexes Catalogs, Categories and SellableItems to help search items in Business tools. The settings for the Search can be found in PlugIn.Search.PolicySet-1.0.0.json

These indexes has a search scope, for searching catalog and related items by default the scope is “xc910CatalogItemsScope”

So in SOLR indexed items related to catalogs are kept in core – “xc910CatalogItemsScope”

By default Sitecore will index only Habitat Environment in Sitecore Commerce 9.1. If you want to index your custom environment or AdventureWorks environment, follow these steps-

  • Open Sitecore.Commerce.Engine.Connectors.Index.Common.config file. This should be in CM Y.Commerce.Engine folder
  • In this instance for Adventureworks add “AdventureWorksAuthoring” in Environemnts element for Master Index and “AdventureWorksShops” for Web Index
  • Open Postman and Run – Full Index Minion Catalog Items
  • Check if the items are indexed correctly in SOLR.
  • Search the same keyword in Business tool for AdeventureWorks environment

First Steps | Sitecore Commerce Development | Create a Custom Plugin

In my last post, I have described the steps to set up the Sitecore Commerce development environment. In this post, I will describe how to create a custom plugin in Sitecore Commerce. I was working on version 9.3 but these steps should work with all versions of 9 series.

Sitecore Commerce provides an extensible framework which can be extended using plugins. A plugin is an independently publishable extension to the Sitecore Commerce Engine. Generally, you will find the following contained in a plugin and are used to extend the platform.

  • Entities
  • Components
  • Entity Views
  • Pipeline
  • Policies etc.

You can extend the platform or add more features using the plugins. Let’s get started and look into the steps to create a new plugin in Sitecore Commerce 9.3. If you have already set up your commerce development environment, please skip to step 2

Step 1 – Setup developer environment

Create a developer environment for Sitecore Commerce to run the engine from Visual Studio, can be either VS 2017 or 2019. Follow this blog post for same Setup development environment for Sitecore Commerce 9.3 Engine

Step 2 – Install Visual Studio Extension for creating plugin

  1. Go to  solution root folder or SDK folder and run the Sitecore.Commerce.Plugin.vsix file. Close the Visual Studio solution if open.
  2. This should install extension on selected Visual Studio Version
create plugin1

Step 3 – Create a plugin

  1. Open your solution. Create a Feature solution folder, just to follow Helix best practice. Right click solution folder to create a new project
  2. Search Sitecore.Commerce.Plugin. Select project type and click next.
create plugin2
  1. Give the project name, location and select .Net Framework 4.7.1. Create a Project.
create plugin3
  1. Sample Folder and Files for Commands, Components, Controller etc shall be added.
create plugin4
  1. Update the Sitecore Commerce Core from 5.0.0 to 5.0.4
create plugin5
create plugin6

4. Build the project. There shouldn’t be any reason the build should fail.

Step 4 – Reference project and run engine

  1. Add Plugin project reference to your Commerce.Engine Project
  2. Run the Engine. You should be able to see the newly created plugin is now registered in Engine.

This should allow to start customizing commerce and truly use plug-gable and extendable feature of Sitecore Commerce.

create plugin9

We intentionally didn’t add any code to this plugin to keep this simple. In the next blog, we will cover the anatomy of the newly created plugin and show how to inject this plugin into Commerce Pipelines and execute some custom code.

Next understand the Elements of Sitecore Commerce Plugin project templates

Hope you enjoyed this post. Stay tuned.

Sitecore 9.3 installation error- Install-SitecoreConfiguration : The ‘Install-SitecoreConfiguration’ command was found in the module ‘SitecoreInstallFramework’, but the module could not be loaded. For more information, run ‘Import-Module SitecoreInstallFramework’.

Whilst installing Sitecore 9.3 received following error-

 

Install-SitecoreConfiguration -Path .\prerequisites.json
Install-SitecoreConfiguration : The 'Install-SitecoreConfiguration' command was found in the module
'SitecoreInstallFramework', but the module could not be loaded. For more information, run 'Import-Module
SitecoreInstallFramework'.
At line:1 char:1
+ Install-SitecoreConfiguration -Path .\prerequisites.json
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (Install-SitecoreConfiguration:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CouldNotAutoloadMatchingModule

SC93Installation

To resolve this issue, you need to set execution policy to your machine-

Set-ExecutionPolicy -ExecutionPolicy AllSigned

You may also provide a scope to the current powershell session-

Set-ExecutionPolicy -ExecutionPolicy AllSigned -Scope Process

This should help resolve issue and any further powershell script execution process depending on what scope you provide the script to run.

References – Set Execution Policy

 

 

 

Sitecore commerce – No catalog data available

Sitecore Storefront can have multiple catalogs, but only one catalog can be used to each Storefront.

At times whilst development you might have to switch between the Authoring site and Visual Studio Commerce Engine Solution for debugging. I have seen sometimes the Catalog for the Storefront is reset during the switch which results in error – No catalog data available.

This message is shown on the menu as seen in the screenshot-

ErrorNoCatalogDataAvailable

ErrorNoCatalogDataAvailable2

Solution-

Login to Sitecore Client and follow these steps-

  1. Navigate to following location in Content Editor-  /sitecore/content/Sitecore/Storefront/Home/CatalogsSelectCatalog
  2. If none of the Catalog is selected, please select the Catalog and Save. In this case Habitat_Master
  3. Should now display the menu as the Catalog details are now setupErrorNoCatalogDataAvailable1

You also might need to consider checking Catalog Configuration here – /sitecore/content/Sitecore/Storefront/Settings/Commerce/Catalog Configuration

Setup the Catalog and Start Navigation Category

CatalogConfiguration

 

Hope this helps.

 

 

Sitecore Commerce environments- configure and access environment policy

Environments in Sitecore XC are used to control the configuration policies that defines behavior. Define a Commerce environment in the environment configuration JSON file. JSON file contains information specific to Commerce Engine policies that needs to be implemented for specific environment for example Habitat Authoring.

In this article will show how to configure the policy in a specific environment JSON file and access the policy that affects the behavior based on the settings.

First check the default environment. Open Sitecore.Commerce.Engine.Connect.config and see

<defaultEnvironment>HabitatAuthoring</defaultEnvironment>

In this case will add a setting to include Sitecore Clinet Url in enviornment-

Open Comerce Authoring – wwwroot\data\Environments folder

Since the default environment is HabitatAuthoring, you should update PlugIn.Habitat.CommerceAuthoring-1.0.0.json file-

{
"$type": "Sample.Commerce.Plugin.Management.Policies.SitecoreCmsServerPolicy, Sample.Commerce.Plugin.Management",
"ServerHostName": "https://mystorefront.local/"
},

Restart IIS. Hope you already know how to run the Commerce Ops script using postman.

Open Postman, run the GetToken and in SitecoreCommerce_DevOps

run Bootstrap Sitecore Commerce.

This should sync the environment related changes.

Assuming you have a plugin. Create a policy in  Sample.Commerce.Plugin.Management.Policies namespace with the class SitecoreCmsServerPolicy inherited from Policy as below-

using Sitecore.Commerce.Core;

namespace Sample.Commerce.Plugin.Management.Policies
{

public class SitecoreCmsServerPolicy: Policy
{
public string ServerHostName { get; set; }
}
}

You can use the environment configurations by getting the policy as folloes-

var cmsSitecoreServerPolicy = context.GetPolicy<SitecoreCmsServerPolicy>();

cmsSitecoreServerPolicy.ServerHostName;

You should now get the value “https://mystorefront.local/&#8221; when ServerHostName is accessed.

You may update these environment files- PlugIn.Habitat.CommerceShops-1.0.0.json for shops environment

Hope this helps.

 

Reference- Commerce Engine environments