Author: sandeeppote

Cache Busting and Version Pinning when building Docker images

docker file – Layered Architecture

Docker uses Layered Architecture. When using Docker files it creates a new layer in the image which adds additional space to the image based on the instructions for that layer.

When a Docker build command is run it proceeds from the first instruction in Docker file to the last while caching each stage so as if the build fails next time build uses cache until it ran succesully and invalidated the stage that failed and the following stage. Layers repurpose the previous layers and don’t have to build all of them again.

In below example Docker file has 6 stages. Each stage will be cached when build command is ran.

Suppose a build fails at Stage 3 due to some reason or new package has to be added the Docker will invalidate the Stage 3 and the following stages

Next time when a issue is rectified the build command will repurpose the previuos layers and build the failed stages

docker file – Layered Architecture

But in this case the repository will not be update, so how to resolve or update the repository with the packages-

Cache Busting

In this case we can to combine the instructions so the repository is updated along with packages as below

docker file – Cache Busting and Version Pinning

Merging Stage 2 and Stage 3 from the previous docker file in to single instruction will ensure the repository is first udpated and pakages are installed

Merging these stages is called as Cache Busting

Version Pinning

You can also explicity mention the version of package to be installed

In stage 2 docker file is instrcuting to install python3-pip 21.3.1 version

Best Practice-

Instructions which are most frequently modified should be at the bottom of the file and the instructions which are least modified should be at the top of the docker file

Docker storage for Ubuntu

Docker uses storage drivers to store the read-only images and writable containers

It basically has 6 layers

Read-only/Image Layers

  1. Base Image e.g. Ubuntu OS
  2. Packages/Repositories e.g. apt etc
  3. Dependencies e.g. pip etc
  4. Custom Code e.g. python code etc
  5. Enrtypoint or command i.e. excutes the program

Writable Layer

6. Container Layer

Layers of a container based on the Ubuntu image

Data and files related to images and containers are store in /var/lib/docker folder in Ubuntu

To check the storage driver used by the docker, use following command-

docker info | more

Im my case it is overlay2

You can also use this command to get the storage driver

docker info | grep "Storage Driver"

How to change the storage driver

Stop the Docker service

systemctl stop docker.socket
systemctl stop docker

Check the docker service status

service docker status

Backup the docker folder

cp -au /var/lib/dovker /var/lib/docker.bk

Change the storage driver

echo '{ "storage-driver": "aufs" }' | sudo tee /etc/docker/dameon.json
service docker start

Image credit and reference links –

https://docs.docker.com/storage/storagedriver/

https://docs.docker.com/storage/storagedriver/overlayfs-driver/

Error while starting the SOLR

Do you see this error while starting the SOLR?

Check the Environment Variable JAVA_HOME and if the path of the jre is correct.

In my case due to the recent JAVA sdk update the path was not changed in environmental variable to reflect the actual path.

So one of the issue could be the incorrect path. Change this to the actual path.

And here we have the SOLR service running

Create your first Sitecore OrderCloud Marketplace

Login to the OrderCloud portal.

https://portal.ordercloud.io/login

You will be shown the Dashboard and an option to create New Sitecore OrderCloud Marketplace

Step 1 – Select Region

While writing this blog there are 4 Regions avaialble for you to select. As per the document you should create Marketplace in Us-West region by default so as to seed using headstart. If you want to use regions other than Us-West you might have to request this to OrderCloud team.

Select region Us-West from the region option.

Step 2 – Autoselected Environment

By default the Sandbox environment is selected for you to try the OrderCloud. If you want to access Staging or Production contact OrderCloud team.

Step 3 – Provide Mareketplace ID [optional]

Provide the Marketplace Id if you want to have your own name. Id’s are Writable i.e. if you chose your ID OrderCloud will generate the same for you or you may let the OrderCloud auto generate the ID.

Chose the Marketplace ID

Step 4 – Provide Marketplace Name

Provide Marketplace name or description here. Click on Create Marketplace.

A new Marketplace will be created and you will be redirected to the Settings tab of the Marketplace with Instance details and other Basic info.

You should also see your newly created Marketplace in Dashboard

Now lets try creating a Marketplace with the same name in the same region i.e. Us-West

You can see OderCloud doesn’t allow to create a new Maretplace with the same name

If you also try creating a Marketplace with the same name in different region it wont allow-

Create a new Marketplace without providing the ID

A new Marketplace is created and OrderCloud gave it a Unique ID

Create a file in Linux

Create a file with touch command

To create a new file, use touch command followed by the name of the file-

This should create a empty file.

touch thirdfile.txt

To create multiple files using touch command-

touch thirdfile-1.txt thirdfile-2.txt

Create a file with cat command

To create a new file with cat command use redirection operator followd by file name.

This will allow to add content to the file

cat > fourthfile.txt

Cratea a file using echo command

To create a new file using echo command use redirection operator followed by file name will create empty file or add content before redirection operator to add content while creating a file.

echo "This is fifth file." > fifthfile.txt

Create a empty file with echo command

echo > sixthfile.txt

Ubuntu – Package Management

DPKG – Debian Package Manager used to install, uninstall, list and check the status of the package

It comes with .deb file.

DPKG does not honour dependecies hence we need to use APT for Ubuntu

APT stands for Advanced Packaging Tool and relies on DPKG

Use apt update command to refresh the repository-

sudo apt update

Use apt upgrade to upgrade exisitng package-

sudo apt upgrade

To install a package-

sudo apt install <package name>

To uninstall or remove package-

sudo apt remove <package name>

To search package e.g:- python-dev

sudo apt search python-dev

To list all the packages in repository-

sudo apt list

Linux File Types

Everything in Linux is file

There are following type of files-

Regular files – Images/scripts/configuration and Data files

Directory – Type of files which saves other files and directories

Special files – have other types

  • Character Files – Represent devices – like Mouse and keyboards
  • Block Files – Represent block devices that writes data in chunk to the devices like HDD and RAM
  • Links – Hard links and Soft Links
  • Socket Files – enables the commnuication between 2 process
  • Named Pipes – passes data from one process to another

Use file command to get the file type-

file <<filename>>

Use ls -ld command to get the file type-

ls -ld firstfile.txt

First character represents the file type

IdentifierFile Type
dDirectory
Regular File
cCharacter Device
lLin
sSocket File
pPipe
bBlock Device

Filesystem Hierarchy

/- Root Partition

/opt – any third party program should be put in this directory

/mnt- mounts the file system from external network temporary to this folder

/tmp- copy any temporary files to this location

/media – copy any media files in this folder

/dev – contains the file character device file like external devices like mouse and keyboard

/bin – basic programs and binaries are located in this directory

/etc – configuration files

/lib and /lib64 – contains shared libraries

/usr – user related data reside in this folder

/var – system writes data such as logs

Linux Runlevels

A runlevel is a categorization number that determines what services are started and
what services are stopped

Runlevel #NameDescription
0HaltAll services are shut down and the server is stopped
1Single User ModeThe root account is automatically logged in to the server and other users cannot log in to the server
2Multiuser ModeUsers can log in to the server through CLI and network service are not started
3Extended Multiuser ModeUsers can log in to the server through CLI and network service are started
4User DefinedUser can cusomize the runlevel
5Graphical ModeUser can log in through CLI and GUI with network services started
6RebootThe server is rebooted

Check the current runlevel use following command-

runlevel

To see the default target

systemctl get-default

To change the default target use following command

systemctl set-default multi-user.target

Commands to understand Linux kernel and hardware

Understand the OS versoin and Linux machine hardware

Use uname command to display information of kernel

uname

For more information of kernel use-

uname -r

5 is Kernel Version

11 is Major Version

0 is Minor Version

1022 is patch release

azure is specific info

To understand CPU of machine use lscpu command

lscpu

In this case Architecture is x86_64 with CPU socket 1 and threads that run in parallel is 4.

To understand physical disk use lsblk command

lsblk

sda shows block devices of type disk

Use lsmem command to get the memory of the Linux machine

lsmem

Use free -m command to get the free memory i.e. total vs used memory

free -m

Append the command with sudo to run the command as super user

Setup Sitecore 10.2 JSS using next js in secured and connected mode

Sitecore 10.2 uses Sitecore Headless Rendering 19.0.0. See link here

See this blog for prerequisites and setting jss using next js in disconnected mode.

Follow below steps after the steps completed in above blog

Download Sitecore Headless Services for Sitecore XM

Prerequisite

Install Sitecore 10.2 XP0 on local development environment. See hte blog here if not already installed

Install downloaded package for Headless Service

https://dev.sitecore.net/Downloads/Sitecore_Headless_Rendering/19x/Sitecore_Headless_Rendering_1900.aspx OR

https://dev.sitecore.net/Downloads/Sitecore_Headless_Rendering/20x/Sitecore_Headless_Rendering_2000.aspx

Run the next js app in connected mode

Before working on connected mode create API Key

Navigate to – /sitecore/system/Settings/Services/API Keys

Create a new API Key e.g.:- nextjsservice

API key here is Item ID

Create a custom binding-

For this create a self signed certificate with a custom domain name in this case – next-jss-prj.dev.local

New-SelfSignedCertificate -CertStoreLocation C:\certificates -DnsName "next-jss-prj.dev.local" -FriendlyName "My First Next JSS App" -NotAfter (Get-Date).AddYears(10)

Create a new https binding and assign the newly create SSL certificate

Add domain to the host file – next-jss-prj.dev.local

Inititate JSS

Run the command

npm init sitecore-jss nextjs

Where would you like your new app created? C:\projects\nextjs
Initializing ‘nextjs’…
What is the name of your app? next-jss-prj.dev.local
What is your Sitecore hostname (used if deployed to Sitecore)? next-jss-prj.dev.local
How would you like to fetch Layout and Dictionary data? REST
How would you like to prerender your application? SSG
Would you like to include any add-on initializers? nextjs-styleguide – Includes example components and setup for
working disconnected
Initializing ‘nextjs-styleguide’…
Which additional language do you want to support (en is already included and required)? da-DK

Setup JSS

jss setup

Is your Sitecore instance on this machine or accessible via network share? [y/n]: y
Path to the Sitecore folder (e.g. c:\inetpub\wwwroot\my.siteco.re):
Invalid input.
Path to the Sitecore folder (e.g. c:\inetpub\wwwroot\my.siteco.re): C:\inetpub\wwwroot\xp102sc.dev.local
Sitecore hostname (e.g. http://myapp.local.siteco.re; see /sitecore/config; ensure added to hosts): next-jss-prj.dev.local
Invalid input. Must start with http(s)
Sitecore hostname (e.g. http://myapp.local.siteco.re; see /sitecore/config; ensure added to hosts): https://next-jss-prj.dev.local
Sitecore import service URL [https://next-jss-prj.dev.local/sitecore/api/jss/import]: https://next-jss-prj.dev.local/sitecore/api/jss/import
Sitecore API Key (ID of API key item): {2318A20A-6DF1-4AB8-B49E-F933A3B79160}
Please enter your deployment secret (32+ random chars; or press enter to generate one):

Deployment secret– Leave this blank as this will generate new secret key

Deployment secret key is generate and stored in <<jss development folder>>\sitecore\config\next-jss-prj.deploysecret.config

JSS connection settings saved to scjssconfig.json

Connection and the deployment secret is created, now this needs to be deployed to Sitecore instance so the JSS app is able to establish a connection with Sitecoore

To deploy the config setting deploy the config to Sitecore instance-

jss deploy config

Configs are copied to Sitecore instance App_Config\Include\zzz…

Deploy the artifacts to Sitecore instance-

Since we don’t know hte thumbprint use the below command with –acceptCertificate as test

jss deploy app –includeContent –includeDictionary –acceptCertificate test

This will provide the thumbprint. highllighted in red

Copy the certificate and try deploying again-

jss deploy app –includeContent –includeDictionary –acceptCertificate E4:54:12:A0:7E:D5:95:9B:B0:C2:00:17:3A:26:1B:AD:71:73:9F:05

if you see this error while importing – Message: An item name must satisfy the pattern: ^[\w*\$][\w\s-\$]*((\d{1,})){0,1}$ (controlled by the setting ItemNameValidation)

Change the ItemnameValidation to .* while importing and revert back to original in Sitecore.config file. Refer this –

https://sitecore.stackexchange.com/questions/470/an-item-name-must-satisfy-the-pattern-w-w-s-d1-0-1-c

To run the app run following command

jss start:connected

If you receive this error-

Execute following command to get this working on port 3000

$env:NODE_TLS_REJECT_UNAUTHORIZED=0

Execute JSS start again-

jss start:connected

ERROR-

Connection to your rendering host failed with a Not Found error. Ensure the POST endpoint at URL http://localhost:3000/api/editing/render has been enabled.

Copy the deploy secret and update the – value of setting of JavaScriptServices.ViewEngine.Http.JssEditingSecret in Sitecore.JavaScriptServices.ViewEngine.Http with the deploy secret.

This now work in connected mode. Any content changes in Sitecore should reflect localhost:3000 and Sitecore instance (https://next-jss-prj.dev.local/)

Sitecore next js working in connected mode with SSL

Error-

Exception thrown while importing JSS app
Exception: System.UnauthorizedAccessException
Message: The current user does not have write access to this item. User: sitecore\JssImport, Item: ContentBlock ({23CE8001-F405-5E50-AE61-AD3057660BBD})
Source: Sitecore.Kernel

Resolution-

Error-

Resolution-

Check if the API key is correct at the location – /sitecore/system/Settings/Services/API Keys

Also check if the CORS and Allowed Controllers are set correctly

For developement environment put * for both the highlighted fields