Kubernetes on Dark Kernel

Implementing Blue-Green Deployments in Kubernetes: Achieving Zero-Downtime Updates

Mon, 15 Jul 2024 00:00:00 +0000

Things we are going to learn

Kubernetes Deployments
Services and Ingress
Database migrations // just as part of the process, not much ;)
Automate with CICD
Monitoring and health checks

Let’s get started.

Blue-Green Deployment Technique

You might or might not know that Blue-green deployment is a technique for releasing applications with zero downtime and easy rollback. Yes, that’s the gist of it, at least conceptually.

This guide will walk you through implementing blue-green deployments in a Kubernetes environment. Yep, we’re diving into Kubernetes.

Implementation Steps

1. Prepare Kubernetes Manifests

Since it is a blue-green deployment, we will need two manifests one for the green version and one for the blue version.

Create separate deployments for blue and green versions:

yaml blue-deployment.yml


apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-blue
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
      version: blue
  template:
    metadata:
      labels:
        app: myapp
        version: blue
    spec:
      containers:
      - name: myapp
        image: myapp:1.0
        ports:
        - containerPort: 8080

green-deployment.yaml (similar, but with version: green and newer image)

yaml green-deployment.yml


# green-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-green
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
      version: green
  template:
    metadata:
      labels:
        app: myapp
        version: green
    spec:
      containers:
      - name: myapp
        image: myapp:1.1
        ports:
        - containerPort: 8080

2. Set Up Service and Ingress

What ingress does is it maps the service to the load balancer, exposing the service externally. The load balancer then distributes traffic between the two deployments.

Create a service that can be switch between blue and green versions:

yaml service.yaml


# service.yaml
apiVersion: v1
kind: Service
metadata:
  name: myapp-service
spec:
  selector:
    app: myapp
    # Note: We don't specify the version (blue/green) here.
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

In the Ingress, we will add the service to the ingress rules.

yaml ingress.yaml


apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "0"
spec:
  rules:
    - host: myapp.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: myapp-service
                port: 
                  number: 80

3. Implement Deployment Script

Automation is the key. Here’s a overview of the bash script to manage the deployment:

#!/bin/bash

# Deploy new (green) version
kubectl apply -f green-deployment.yaml

# Wait for green deployment to be ready
kubectl rollout status deployment/myapp-green

# Run smoke tests on green deployment
# (implement your tests here)

# Switch the traffic to green. Here the game changes
kubectl patch service myapp-service -p '{"spec":{"selector":{"version":"green"}}}'

# Wait and monitor for any issues
sleep 60

# If everything is fine, delete the old (blue) deployment
kubectl delete -f blue-deployment.yaml

# If not, you can quickly rollback by switching the service back to blue
kubectl patch service myapp-service -p '{"spec":{"selector":{"version":"blue"}}}'

These are the steps we need to implement in our deployment script.

4. Database Migrations

Database migrations in a blue-green deployment scenario are challenging because you need to maintain compatibility between the two versions of your application (blue and green) and the database schema. What we want is to perform these migrations without downtime and ensure both versions can operate correctly during the transition. which is damn easy :) (maybe)

You can use any of your strategies. For database changes, you can use a migration tool that supports reversible migrations. *Run migrations before deploying the green version:

# I will be something like this:
./migrate up

# If migration fails, you can roll back
./migrate down

It’s not only this much, but it’s enough for now.

5. Monitoring and Health Checks

Implement readiness and liveness probes in your deployments:

spec:
  containers:
  - name: myapp
    image: myapp:1.0
    readinessProbe:
      httpGet:
        path: /healthz
        port: 8080
      initialDelaySeconds: 10
      periodSeconds: 5
    livenessProbe:
      httpGet:
        path: /healthz
        port: 8080
      initialDelaySeconds: 15
      periodSeconds: 10

Once the application is up and running, possibly if you’re lucky or an expert, you can monitor it with Prometheus and Grafana.

6. Automate with CI/CD Pipeline

Integrate the deployment process into your CI/CD pipeline. Personally, I use GitLab (you’d never guess). Here’s an example GitLab CI configuration:

stages:
  - build
  - test
  - deploy

build:
  stage: build
  script:
    - docker build -t myapp:$CI_COMMIT_SHA .
    - docker push myapp:$CI_COMMIT_SHA

test:
  stage: test
  script:
    - run_unit_tests.sh
    - run_integration_tests.sh

deploy:
  stage: deploy
  script:
    - update_kubernetes_manifests.sh $CI_COMMIT_SHA
    - ./blue_green_deploy.sh
  only:
    - main

And if you are wondering what those scripts contain, Then check this out

Best Practices

Automate everything: From builds to deployments to rollbacks.
Implement robust monitoring: Quickly detect any issues post-deployment.
Gradual rollout: Consider canary releases before full blue-green switch.

Conclusion

In the conlcusion, Kubernetes’s blue-green deployments aren’t just a fancy color scheme—they’re your ticket to stress-free updates. It provide a powerful way to achieve zero-downtime updates. By combining infrastructure management, automation, and *careful testing, we can significantly reduce some of the risks of deployment and improve availability.

And, that’s it.

Kubernetes Voyage: Advanced Ship Steering

Sun, 10 Sep 2023 00:00:00 +0000

We will learn

pods vs container
Deployments
Services
Auto-healing concept
config maps & secrets
persistent volume

Let’s get started.

Pods vs Containers

Pods	Container
Unit of conainers	Single container by containerization tool.
Contains one or more Conainers	Only single container
Crashed free	Can be crashed if some error occurs
Can be Scheduled	Can’t be scheduled

Configuration file

To create pods we have to use a configuration file in YAML, which is also called manifest in k8s

We need manifest file for almost everything in kubernetes.

Kind:

To create maifest file for a paritcular service we have to define a kind keyword. kind defines what type of manifest file is, like for Pods, Service or Deployment, etc.

Example:

apiVersion: v1
kind: Pod
metadata:   # Extra data. # object
  name: nginx  # Any name can be used.
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80

Namespace:

logical entity allow you to isolate your resources, like pods, volumne, deploymnets, etc. You can create multiple namaspcae for different resources, it is a type of group.

Default name space is already present, and everything is created under it until you specify any other namespace in manifest.

kubectl create namespace nginx

kubectl get namespace

Example:

yaml Pod.yml


apiVersion: v1
kind: Pod
metadata:   # Extra data. # object
  name: nginx  # Any name can be used.
  namespace: nginx
spec:
  containers:
  - name: nginx
    image: nginx:latest
    ports: # If you don't  specify then also it will assume.
    - containerPort: 80

To apply the manifest file to your cluster command:
```
kubectl apply -f pod.yml
```
This will send request to api server, then it will to scheduler (resposnisble fo managing pods) and then it send back to api server and then configuration will be stored.
To fetch the pods information from master we can use the below command:
```
kubectl get pods -n nginx
```
-n: namespace, by default it only looks for resources in to default namespace.

Deployment

Deployment is a desired state. It is also written in manifest file and kind will be set to Deployment. It is used to create replicaset of templates, It is a configuration of pods. It is a desired(required) state of your pods. You can also provide some data in pods while deployment using this manifest.

Example:

yaml deployment.yml


apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:  # select all nginx labels
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

replicas: Number of pods clone / replications
labels: Configuration is for, name of the replica.

Actions:

Apply the deployment:
```
kubectl apply -f deployment.yml 
```

Get the nodes with more wider output.

kubectl get nodes -n nginx -o wide

NAME               STATUS   ROLES                  AGE     VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION   CONTAINER-RUNTIME      
ip-172-31-55-54    Ready    <none>                 4d23h   v1.20.0   172.31.55.54    <none>        Ubuntu 22.04.3 LTS   6.2.0-1011-aws   docker://24.0.5        
ip-172-31-57-166   Ready    control-plane,master   5d      v1.20.0   172.31.57.166   <none>        Ubuntu 22.04.3 LTS   6.2.0-1011-aws   docker://24.0.5

Get your deployment status.

kubectl get deployment -n nginx

NAME               READY   UP-TO-DATE   AVAILABLE   AGE                                        │
nginx-deployment   3/3     3            3           23m

Get detail information about your deployment.
```
kubectl describe deployment 
```

Apply Rolling update to deployment like scaling: # No need to edit the file.

kubectl scale deployment nginx-deployment --replicas=2 -n nginx

deployment.apps/nginx-deployment scaled

You can even Rollback if you made any mistake.

kubectl rollout undo deployment/nginx-deployment

Services

It is method used to allow outside world to access application instance running in pods deployment, some sort of proxy.

In our example, nginx is running but not accessible because we haven’t applied any service yet.

Example:

yaml service.yml


apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app.kubernetes.io/name: MyApp
  ports:
    - protocol: TCP
      port: 80 # containers port
      targetPort: 9376  # service's port to map with.

Services are of 3 types:

NodePort:

It act as a node machine, this maps the deployment’s port to serivce’s port.

We have to provide 3 ports:

port: Port of application running in pods, (80 incase of nginx).
targetPort: Service port, to map pods port with Service port. (incoming)
nodePort: Outgoing port, which is actual accessible port. ( ranges [30000 - 32627])

Example:

This service file will provide the access to users for our application which is nginx in this case.

yaml service.yml


apiVersion: v1                                                                                 
kind: Service                                                                                  
metadata:                                                                                      
  name: my-service                                                                             
  namespace: webserverss                                                                       
spec:                                                                                          
  selector:                                                                                    
    app: nginx                                                                                 
  type: NodePort                                                                               
  ports:                                                                                       
    - protocol: TCP                                                                            
      port: 80                                                                                 
      targetPort: 80                                                                           
      nodePort: 30004

Cluster IP:

It Exposes the Service on a cluster-internal IP, Making the Service only reachable from within the cluster not outside, And this is used by default.
Load balancer:

It Exposes the Service externally using an external load balancer. Kubernetes does not offer a load balancing component; you have provide one, or you can integrate your cluster with a cloud provider like AWS.

We, will use NodePort which will allow us to access out nginx outside the cluster.

Apply the service
```
kubectl apply -f serivce.yml
```

List your Services.

kubectl get services -n webserverss

NAME         TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE                       │
my-service   NodePort   10.104.86.172   <none>        80:30004/TCP   15s

Auto healing.

Now, our pods are running and our application is also accessible by the user. But, what if we delete one of the pod or container?

Well, let’s do it.

First get list of your pods.

kubectl get pods -n webserverss # replace with your namespace

NAME                                READY   STATUS    RESTARTS   AGE                           
nginx                               1/1     Running   1          17m                         
nginx-deployment-66b6c48dd5-2299p   1/1     Running   1          16m                         
nginx-deployment-66b6c48dd5-vtqq8   1/1     Running   1          16m

And now delete one pod.

kubectl delete pod nginx-deployment-66b6c48dd5-2299p -n webserverss

Now, we have deleted one pod, let’s check the status of pods.

NAME                                READY   STATUS    RESTARTS   AGE                         
nginx                               1/1     Running   1          17m                         
nginx-deployment-66b6c48dd5-tlq9v   1/1     Running   0          8s                            
nginx-deployment-66b6c48dd5-vtqq8   1/1     Running   1          16m

You saw that right? The Age of second pod is 8s.

Yes, that 2nd pod got created after we deleted one, And this is what we call Auto Healing.

Anyway, If you want to delete you particular depolyment or manfiest configuration use this syntax:
```
kubectl delete -f pod.yml # your manifest.yml file
```

Secrets & config map

In k8s, we can’t pass environment variables through kubectl at runtime. Therefore we use secret & config map file.

Config map:

It is a Special kind of manifest file, if you deployment needs particular variable then you can declare that in this manifest file. All variables are passed to all pods. In manifest set kind to kind: ConfigMap to create a config map. It pass data as plain texts.
Secrets:

It is a type of config which contains credentials (passwords), advantage is you can pass encoded password and it will be decrypted in pods. In manifest set kind to kind: secrets to create secrets.

Example:

Setup servers

Create deployment manifest.

yaml deployment.yml


apiVersion: apps/v1                                                                               
kind: Deployment                                                                                  
metadata:                                                                                         
  name: mysql-deplyment                                                                           
  namespace: mysql                                                                                
  labels:                                                                                         
    app: mysql                                                                                    
spec:                                                                                             
  replicas: 1                                                                                     
  selector:                                                                                       
    matchLabels:                                                                                  
      app: mysql                                                                                  
  selector:                                                                                       
    matchLabels:                                                                                  
      app: nginx-2                                                                                
  template:                                                                                       
    metadata:                                                                                     
      labels:                                                                                     
        app: nginx-2                                                                              
    spec:                                                                                         
      containers:                                                                                 
        - name: mysql                                                                             
          image: mysql                                                                          
          ports:                                                                                  
            - containerPort: 3306                                                                 
          env:                                                                                    
            - name: MYSQL_DATABASE                                                                
              valueFrom:                                                                          
                configMapKeyRef:                                                                  
                  name: mysql-config                                                              
                  key: MYSQL_DATABASE                                                             
              name: MYSQL_ROOT_PASSWORD                                                           
              valueFrom:                                                                          
                secretKeyRef:                                                                     
                  name: mysql-secret                                                              
                  key: MYSQL_PASSWORD

You can rectify errors using dry run, it doesn’t apply the changes only show what will be happen if applied:
```
kubectl apply -f deployment.yml --dry-run
```
Create defined namespace:
```
kuebctl create namespace mysql
```

Create configMaps manifest file

yaml configMaps.yml


apiVersion: v1
kind: ConfigMap
metadata: 
    name: mysql-config
    namespace: mysql
    labels:
        app: mysql
data: 
    MYSQL_DATABASE: "cooldb"

Apply config map.
```
kubectl apply -f configMap.yml
```
Check your config map.
```
kubectl get configmap -n mysql
```

Now create secrests manifest.

But, before that encrypt your pass using below command.

echo -n 'secret' | base64  # you can pass -d to decrypt

copy output and pass in data field in secrets.yml

yaml secrets.yml


apiVersion: v1
kind: Secret
metadata:
    name: mysql-secret
    namespace: mysql
    labels:
        app: mysql
type: Opaque
data: 
    MYSQL_PASSWORD: dkoafusfmsdjk # base64 password.

Apply secrets manifest to deployment.
```
kubectl apply -f secret.yml
```
Now, check if secret is added successfully.
```
kubectl get secret -n mysql
```

After that, finally apply your deployment.

kubectl apply -f deployment.yml

NAME              READY   UP-TO-DATE   AVAILABLE   AGE                                            │
mysql-deplyment   1/1     1            1           40m

After your deployment, you can cross verify by logging in mysql. Run commands in worker node. Make sure you are putting correct container id.

docker ps | grep mysql 
docker exec -it ef175a2293f2 /bin/mysql -u root -p

You can delete your deployment if you want.
```
kubectl delete -f deployment.yml
```

PVC & PV, Storage classes

PV (Persistent Volume)

It creates application’s stateful location is in your disk where data is stored as snapshot.

yaml persistenVolume.yml


apiVersion: v1
kind: PersistentVolume
metadata: 
    name: mysql-pv-volume
    namespace: mysql
    labels:
        app: mysql
spec: 
    storageClassName: manual
    capacity: 
        storage: 2Gi
    accessModes: 
        - ReadWriteOnce
    hostPath: 
        path: "/mnt/data"

Make sure hostPath.path directory exists in worker node.

Now apply your peristentVolume manifest

kubectl apply -f persistenVolume.yml

Check if persistentVolume is created.

kubectl get pv -n mysql

NAME              CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE                                                                                        
mysql-pv-volume   2Gi        RWO            Retain           Available           manual                  16s

PVC (Persistent Volume Claim):

After creating the volume you have to claim it to use, until now we have just created not used. You can claim how much you want from volume for your deployment.

yaml persistentVolumeClaim.yml


apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
  namespace: mysql
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

Then, apply it

kubectl apply -f persistentVolumeClaim.yml

After that check if was successfull.

kubectl get pvc -n mysql

Now, to use volumes add these into containers object in deployment.yml.

        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pv-claim

So, that’s it.

Kubernetes Ahoy - Start Steering Your Container Ship

Sun, 27 Aug 2023 00:00:00 +0000

We will learn,

Kubernetes Architecture
Deployment of first cluster.

Kubernetes

Kubernetes, also known as K8s, is an open-source container orchestration platform system for automating deployment, scaling, and management of containerized applications.

Key Features:

Auto scaling (Scale according to traffic)
Auto healing (Ex. Even if we kill one container, It will maintain the no. of container and will create new one.)

Application Architectures

Monolithic:

It is Solo / single structure, Whole Application Architecture on single codebase and runtime environment.

Disadvantage: If you want to change something in a part then your whole application will be affected.
Microservice:

Single application is deployed as different microservices like, frontend, backend, DB, etc. Use different services, All services are interconnected, if you want to change in some service you can change and no need to stop whole application and neither it will affected.

Example : Kubernetes, openshift, serverless.

Kubernetes Architecture

Kubernetes is consists of two nodes master & worker, Nodes are the physical or virtual machines that are used to run pods.

There can be only one master and multiple worker nodes.

Kubernetes architecture

Master: (Control plane)

Master node often reffered to as control plane and responsible for managing and orchestrating the overall operations of the system, It serves as central control point of cluster. It interact with worker node to deploy pods.

cluster is a group of servers. In kubernetes it refers to entire set of computing resources and components that collectively make up a Kubernetes environment, like Pods, worker node, master node, services, etc.

Components:
- Schedhuler: schedule worker node for running pods.
- Controller manager: The main function is to maintain desired state of cluster. Checks what the workers are doing and they are up.
- API server: Directly communicated with worker from master and vice versa.
- etcd: Store state of kubernetes cluster in key-value data store.
- Kubectl: Kubectl is a command-line tool used to communicate with a Kubernetes cluster’s control plane using the API server. Allows to run commands to deploy application, inspect and manage resources.
- CNI: Container network interface.
- Pod: This host and manage our containers that run our application.
Worker: Machine which runs containers and workloads. This is where the actual application is running.

Components:
- kubelet: Stay in worker node, Manage containers and ensures they’re running as expected. It communicates with the API server to receive information about the pods that are assigned to the node.
- service proxy : It maintains some network rules which determines how traffic is allowed to and from the Pods. It also Allow users/clients to access application.

Now, Let’s run.

Required tools:

Minikube: Used to create local cluster, works in single server using virtualiztion (docker, vmware, virtualboxdocker, vmware, virtualbox).
kubeadm: Used in production servers. (We will be using this)
CRI: Container runtime interface. (Docker, podman, etc.)
Servers: 2 machines, with at least 2G of ram and 2 cpu. (you can use t2.medium if you are using AWS)

After creating servers execute commands according on master and worker:

Install Dependencies # Execute On both the nodes (master and worker)

sudo apt update
sudo apt-get install -y apt-transport-https ca-certificates curl
sudo apt install docker.io -y
sudo systemctl enable --now docker

Install kubeadm # Execute On both the nodes (master and worker)

curl -fsSL "https://packages.cloud.google.com/apt/doc/apt-key.gpg" | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/kubernetes-archive-keyring.gpg
echo 'deb https://packages.cloud.google.com/apt kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt update
sudo apt install kubeadm=1.20.0-00 kubectl=1.20.0-00 kubelet=1.20.0-00 -y

Now, initialize kubeadm (kubernetes) in Master node.
```
sudo kudeadm init
```

Setup local kubeconfig # Execute on master node.

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Apply weave network # On master

kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml

Generate token for worker node to join # On master
```
sudo kubeadm token create --print-join-command
```
Now, copy the output and we will use it in worker node.
Reset the kubeadm on worker node
```
sudo kubeadm reset pre-flight checks
```

Now, paste the token (output) of 6th step, also append --v=5 at end.

Example:

sudo kubeadm join 172.31.61.228:6443 --token f4mesu.7rzk86ga48n3uydh --discovery-token-ca-cert-hash sha256:66c9863913ffdb50316e82b74f3703f73e42c4210d3a01ec7afcdbc01f677eec --v=5

After successful connection you will see output similar to this:

This node has joined the cluster:
  * Certificate signing request was sent to apiserver and a response was received. 
  * The kubelet was informed of the new secure connection details.

Verify you worker node connection by running this command on Master node.
```
kubectl get nodes
```

You can now run test application on pod. # On master

kubectl run hello-world-pod --image=busybox --restart=Never --command -- sh -c "echo 'Hello, World' && sleep 3600"

And verify using docker ps -a command on worker node.

You can even run nginx image

kubetctl run nginx --image=nginx

Check if pod is created in worker node:

kubectl get pod

So, that’s it for getting started.

Kubernetes on Dark Kernel

Implementing Blue-Green Deployments in Kubernetes: Achieving Zero-Downtime Updates

Blue-Green Deployment Technique

Implementation Steps

1. Prepare Kubernetes Manifests

2. Set Up Service and Ingress

3. Implement Deployment Script

4. Database Migrations

5. Monitoring and Health Checks

6. Automate with CI/CD Pipeline

Best Practices

Conclusion

Kubernetes Voyage: Advanced Ship Steering

Pods vs Containers

Configuration file

Deployment

Services

Services are of 3 types:

Auto healing.

Secrets & config map

PVC & PV, Storage classes

Kubernetes Ahoy - Start Steering Your Container Ship

Kubernetes

Application Architectures

Monolithic:

Microservice:

Kubernetes Architecture

Master: (Control plane)

Worker: Machine which runs containers and workloads. This is where the actual application is running.

Now, Let’s run.