Pointers for Automating Kubernetes Deployments

Kubernetes is the market's leading open-source tool for automating deployment, scaling, and management of containerized applications. Most commonly, it's used with Docker containers. All major cloud providers have Kubernetes hosted solutions, like AWS-EKS, GCP-GKE, and Microsoft AKS.  For more information, please refer to the Kubernetes docs .

Kubernetes works well if teams are using microservice/SOA-based architectures. It's also helpful if teams have access to a K8s ( short for Kubernetes) development clusters to develop and test there changes.

k8s Orchestration

Most cloud providers have automated one-click installation for Kubernetes clusters, and k8s (short for Kubernetes) can also be hosted on on-premise infrastructure . Terraform can also be used to automate your k8s infrastructure .

Another good option is KOPS , which is wonderful at orchestrating production-grade clusters; you can also use it for k8s development/QA/prod environments.

Step 1: SCM Strategy

It goes without saying, but you should have a planned strategy, like developing a branch pointing to a dev k8s cluster and the master branch to the prod cluster, etc. I don't suggest using Kubernetes Namespaces for environment promotion paths, especially if you are at the initial stage of your Kubernetes development. 

Teams need to have a clear promotion path and no shared networking on resources between d ev and prod clusters. Keep the production environment isolated to reduce your blast radius. Use automated testing to validate automated promotion across environments

Step 2: CI Orchestration and k8s Packaging

You will need a CI orchestration tool like Jenkins, Travis CI, or Circle CI to automate your deployment pipelines. Helm is one of the most popular packaging formats for Kubernetes.

Plan on versioning your Helm charts for each microservice and auto-updating values.yaml files. Helm packaging is supported on Artifactory, Quay, S3, GitHub, and more.

Here's the code snippet for the Jenkinsfile:

stage('Deploy to K8s-Dev') {
   when {
           branch 'develop'
         }
            steps {
                helm upgrade --install --force --namespace $NAME_SPACE $APP_NAME ./helm/${APP_NAME} --set image.tag=$VERSION
                helm package ./helm/${APP_NAME}
                curl -ujenkins:${JKS_TKN} -T ${APP_NAME}-${VERSION}.tgz "${HELM_UPLOAD_URL}/${APP_NAME}-${VERSION}.tgz"
            }
         branch 'master'
            steps {
                 ...
                   }

Have some automated steps to validate each deployment, as in this code snippet:

stage('Acceptance-Test') {
            when {
                branch 'master'
            }
            steps {
                script {
                    sleep (time: 5)
                    def response = httpRequest (
                        url: "http://$APP_URL:8081/",
                        timeout: 30
                    )
                    if (response.status != 200) {
                        error("Acceptance test failed - Deployment failed.")
                    }
                }
            }
        }

Step 3: Kubernetes Architecture

Plan your Kubernetes architecture:

1. Ingress controller (Istio, etc.):Define each service type (ClusterIP, Nodeport, or load balancer). Plan on using an ingress controller like Istio in the development environment. It provides lots of routing and monitoring capabilities out-of-the-box, but currently, most features are still in alpha and beta so I won't recommend it to use in production environments.

2. Secrets (certificates, user credentials):Certificates and secrets are used for things like downloading artifacts from a private registry, connecting to databases, and securing your services. Plan your a rchitecture for s calability and security; it's a bad idea to put your secret configurations in the source code.

3. Namespace mapping:Plan in advance how you want to use Kubernetes Namespaces. You can define namespaces as per the environment progression or separate namespaces for each application. 

4. ConfigMaps:  ConfigMaps are used to separate configuration artifacts from image content to keep containerized applications portable. Using ConfigMaps, we can pass configuration details as key-value pairs which can be later consumed by pods or any other system components, such as controllers.

Step 4: Managing Endpoints and Base Infrastructure

It's a pretty common requirement for a microservice running inside Kubernetes to use infrastructure hosted outside of k8s, like databases, message queues (Kafka/Zookeeper), etc.  If your company is not ready for a full migration to a container  orchestration tool like Kubernetes, you can use service endpoints to connect to your base infrastructure.

Here is an example of how to connect a private DB using endpoint services and Private-IP. You can use a similar configuration to connect to external Kafka/Zookeeper or a mainframe system.

kind: Service
apiVersion: v1
metadata:
 name: private-db
Spec:
 type: ClusterIP
 ports:
 - port: 3000
   targetPort: 3000
--
apiVersion: v1
kind: Endpoints
metadata:
  name: private-db
subsets:
- addresses:
  - ip: 10.16.1.10
  ports:
  - port: 3000
    protocol: TCP

Please notice that in the service section, there are no pod selectors. This creates a service, but it doesn’t know where to send the traffic. We manually create endpoint objects that will receive traffic from this service.

You can now connect your microservices to the private database like so:

private-db://private-db

kubectl get endpoints private-db
NAME        ENDPOINTS          AGE
private-db  10.16.1.10:3000    1d

Mapping external services to internal ones gives you the flexibility to use hybrid infrastructure for now and also give you the flexibility to bring these external services into the cluster in the future.

Do not use an endpoint address as a ConfigMap or use it in your code as an environment variable. This will work, but it will make your application flaky and unstable, and you will have to re-deploy your application every time your endpoint IP changes.

Conclusion

There is so much more to a stable Kubernetes architecture, like implementing security and monitoring, and this article is in no way a definite guide for Kubernetes migration. I just want to give teams planning to migrate to Kubernetes some options and pointers.