Tag: Ubuntu

Setup Sitecore OrderCloud headstart with Angular in Ubuntu system the docker way

Working on Linux system after been worked on Windows for 2 decades is always fun.

This blog post is to setup the Sitecore OrderCloud Headstart on Ubuntu the Docker way. As all the images used for the Headstart is are Linux based, this I didn’t find major difference how this is been setup in Windows system apart from few changes while installing Storage Explorer and few other erros which I have noted in this blog post.

Note – use sudo for each command or “sudo i”to run as root.

Ensure node js is installed

This might be required for your local build.

sudo apt update

sudo apt install nodejs

Ensure npm is installed

sudo apt install npm

Ensure docker and docker -compose is installed

See this blog post Install Docker on Linux

sudo snap install docker

sudo apt install docker-compose

Docker Compose

Lets start composing and solve errros that might come ourr way-

sudo docker-compose up -d

npm needs TLS1.2

npm notice Beginning October 4, 2021, all connections to the npm registry – including for package installation – must use TLS 1.2 or higher. You are currently using plaintext http to connect. Please visit the GitHub blog for more information: https://github.blog/2021-08-23-npm-registry-deprecating-tls-1-0-tls-1-1/
npm WARN @ordercloud/headstart-sdk@0.0.0 No repository field.

https://stackoverflow.com/questions/69044064/npm-notice-beginning-october-4-2021-all-connections-to-the-npm-registry-incl

npm cache clear --force

npm set registry=https://registry.npmjs.org/

npm install -g https://tls-test.npmjs.com/tls-test-1.0.0.tgz

Install .Net SDK

Middleware runs on .Net. So this neds to be installed.

https://learn.microsoft.com/en-us/dotnet/core/install/linux-ubuntu

https://devblogs.microsoft.com/dotnet/dotnet-6-is-now-in-ubuntu-2204/

sudo apt-get update && \
sudo apt-get install -y dotnet-sdk-6.0
sudo apt-get update && \
sudo apt-get install -y aspnetcore-runtime-6.0
sudo apt install dotnet6

I found difficulties installing .Net on Ubuntu. You may have todo few restarts.

 docker compose up -d

This should start all the containers. Note- cosmos container takes time to start till then middleware waits and starts when comos is ready.

Comos should be available now –

Install Azure Storage- Explorer

Install Storage explorer in Ubuntu-

snapd should be already installed if you are using Ubuntu 16.04 LTS or later, you may have to update.

sudo apt update
sudo apt install snapd

Install Storage Explorer

sudo snap install storage-explorer

Open Azure Storage Explorer and follow the steps here –

Execute the command-

snap connect storage-explorer:password-manager-service :password-manager-service

Azure Storage Explorer should be able to open with above command.

Apply the same settings mentioned in this blog

Once you have followed and applied the settings mentioned in the blog, should be able to see the translation files in local storage and able to access the file.

We also have to set CQRS for blob container – lets do this later.

Middleware exited with errors-

Error –

See the resolution to this issue here – section – Unable to start Middleware container due to erros

Error – Connection refused (127.0.0.1:8081)

System.AggregateException: One or more errors occurred. (Connection refused (127.0.0.1:8081))
       ---> System.Net.Http.HttpRequestException: Connection refused (127.0.0.1:8081)

See the resolution to this issue here – section – Connection to Comos DB is failing

Error- Unsupported platform

0 18.52 npm ERR! code EBADPLATFORM
#0 18.53 npm ERR! notsup Unsupported platform for fsevents@2.3.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
#0 18.53 npm ERR! notsup Valid OS:    darwin
#0 18.53 npm ERR! notsup Valid Arch:  any
#0 18.53 npm ERR! notsup Actual OS:   linux
#0 18.53 npm ERR! notsup Actual Arch: x64

Changed the node version- see the blog here

Also changed the nginx version – see Error 4 here

 => ERROR [headstart-seller:local-linux production 4/8] RUN apk add --update nodejs nodejs-npm && npm install -g json                                                                                 1.0s

see Error 4 here

Change from this  -
RUN apk add --update nodejs nodejs-npm && npm install -g json

to-
RUN apk add --update nodejs npm && npm install -g json
#0 51.97 npm ERR! npm ERR! Cannot read properties of null (reading 'pickAlgorithm')

See the resolution to this error here

Now you should have all containers up and running with

sudo docker compose up -d

If you see this error-

Check for Configure CORS to Blob Containers in this blog post

And here I have Seller, buyer and middleware working on Ubuntu system-

This has really opened the horizon to develop, deploy and maintain OrderCloud solution on a technology agnostic platform.

Loading

Linux and Docker on Ubuntu Series

Linux basic commands (Ubuntu)

Linux Kernel and hardware

Linux Runlevels

Linux Package Management for Ubuntu

Linux User Management commands

Linux Networking commands

Install Docker on Linux

Linux File Types

Create a file in Linux

How to check the file size in Linux

Compressing and Uncompressing Files in Linux

Searching files and directories in Linux

Search content with pattern in the file in Linux

Search content with pattern in the file in Linux

File Permissions in Linux

Check running services in Linux

DOCKER

Docker FAQ’s

Install Docker on Ubuntu

Install Docker using install script on Ubuntu

Setup a Docker Swarm

Cache Busting and Version Pinning when building Docker images

Docker storage on Ubuntu

How to start docker in debug mode in Ubuntu

Docker Restart Policies

Use Docker image offline with Save and Load command in Ubuntu

Export Container and Import as Image using Docker in Ubuntu

Create a custom network in docker for communication between containers

Dcoker Security

Docker Best practice

Use Docker Image offline with Save and Load command in Ubuntu

Docker Save and Load command

At times if you dont want to always pull the image from image registry which takes time to pull if the image’s are heavy, it makes sense to save the image and use it offline. This avoids to always pull the image from the registry.

Save the image once into tar file and reuse the images.

To Save image use follwoing command

First pull the image from repository

docker pull httpd
docker image save httpd -o httpdimage.tar

Get the image from the tar file instead pulling it from the registry

docker image load -i httpdimage.tar

Cache Busting and Version Pinning when building Docker images

docker file – Layered Architecture

Docker uses Layered Architecture. When using Docker files it creates a new layer in the image which adds additional space to the image based on the instructions for that layer.

When a Docker build command is run it proceeds from the first instruction in Docker file to the last while caching each stage so as if the build fails next time build uses cache until it ran succesully and invalidated the stage that failed and the following stage. Layers repurpose the previous layers and don’t have to build all of them again.

In below example Docker file has 6 stages. Each stage will be cached when build command is ran.

Suppose a build fails at Stage 3 due to some reason or new package has to be added the Docker will invalidate the Stage 3 and the following stages

Next time when a issue is rectified the build command will repurpose the previuos layers and build the failed stages

docker file – Layered Architecture

But in this case the repository will not be update, so how to resolve or update the repository with the packages-

Cache Busting

In this case we can to combine the instructions so the repository is updated along with packages as below

docker file – Cache Busting and Version Pinning

Merging Stage 2 and Stage 3 from the previous docker file in to single instruction will ensure the repository is first udpated and pakages are installed

Merging these stages is called as Cache Busting

Version Pinning

You can also explicity mention the version of package to be installed

In stage 2 docker file is instrcuting to install python3-pip 21.3.1 version

Best Practice-

Instructions which are most frequently modified should be at the bottom of the file and the instructions which are least modified should be at the top of the docker file

Docker storage for Ubuntu

Docker uses storage drivers to store the read-only images and writable containers

It basically has 6 layers

Read-only/Image Layers

  1. Base Image e.g. Ubuntu OS
  2. Packages/Repositories e.g. apt etc
  3. Dependencies e.g. pip etc
  4. Custom Code e.g. python code etc
  5. Enrtypoint or command i.e. excutes the program

Writable Layer

6. Container Layer

Layers of a container based on the Ubuntu image

Data and files related to images and containers are store in /var/lib/docker folder in Ubuntu

To check the storage driver used by the docker, use following command-

docker info | more

Im my case it is overlay2

You can also use this command to get the storage driver

docker info | grep "Storage Driver"

How to change the storage driver

Stop the Docker service

systemctl stop docker.socket
systemctl stop docker

Check the docker service status

service docker status

Backup the docker folder

cp -au /var/lib/dovker /var/lib/docker.bk

Change the storage driver

echo '{ "storage-driver": "aufs" }' | sudo tee /etc/docker/dameon.json
service docker start

Image credit and reference links –

https://docs.docker.com/storage/storagedriver/

https://docs.docker.com/storage/storagedriver/overlayfs-driver/

Ubuntu – Package Management

DPKG – Debian Package Manager used to install, uninstall, list and check the status of the package

It comes with .deb file.

DPKG does not honour dependecies hence we need to use APT for Ubuntu

APT stands for Advanced Packaging Tool and relies on DPKG

Use apt update command to refresh the repository-

sudo apt update

Use apt upgrade to upgrade exisitng package-

sudo apt upgrade

To install a package-

sudo apt install <package name>

To uninstall or remove package-

sudo apt remove <package name>

To search package e.g:- python-dev

sudo apt search python-dev

To list all the packages in repository-

sudo apt list

Linux Runlevels

A runlevel is a categorization number that determines what services are started and
what services are stopped

Runlevel #NameDescription
0HaltAll services are shut down and the server is stopped
1Single User ModeThe root account is automatically logged in to the server and other users cannot log in to the server
2Multiuser ModeUsers can log in to the server through CLI and network service are not started
3Extended Multiuser ModeUsers can log in to the server through CLI and network service are started
4User DefinedUser can cusomize the runlevel
5Graphical ModeUser can log in through CLI and GUI with network services started
6RebootThe server is rebooted

Check the current runlevel use following command-

runlevel

To see the default target

systemctl get-default

To change the default target use following command

systemctl set-default multi-user.target

Install Docker on LINUX – Ubuntu

OS requirements

To install Docker Engine, you need the 64-bit version of one of these Ubuntu versions:

  • Ubuntu Hirsute 21.04
  • Ubuntu Groovy 20.10
  • Ubuntu Focal 20.04 (LTS)
  • Ubuntu Bionic 18.04 (LTS)

To check the OS version of Linux, use-

cat /etc/*release*

Check if you have already install docker on machine-

docker --version

Uninstall the older version of Docker if you have any, to do so-

sudo apt-get remove docker docker-engine docker.io containerd runc

Install Repository

Before the docker is installed setup the docker repository-

User Application Package Tool (APT) to install supporting packages for Docker. Use following commands-

sudo apt-get update
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

Add Docker’s official GPG key:

GPG or GNU Privacy Guard is a public key cryptographic implementation and is used for allowing secure transmission of information between source and destination

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

Install Docker Engine

sudo apt-get update

sudo apt-get install docker-ce docker-ce-cli containerd.io

This installs the latest version of docker on OS.

Check the latest version of docker

docker --version

Reference documents- https://docs.docker.com/engine/install/ubuntu/