A GitOps development environment in the comfort of your own localhost

All great frameworks have easy to use local development environments. Similarly, the Kubestack GitOps framework provides a local development environment for infrastructure automation. This tutorial will help you setup a complete GitOps lab on your local machine. Including Kubernetes nodes running as Docker containers using Kubernetes in Docker (KinD) and optionally even a local pipeline triggered using Git hooks.

Having a local lab can be useful to:

• Learn more about GitOps.
• Try the Kubestack framework.
• Use the lab as a local development environment.

Following this tutorial you will:

1. Bootstrap your local GitOps repository.
2. Configure your lab infrastructure.
3. Provision the local clusters.
4. Optional: Test the GitOps flow.

The KinD starter comes pre-configured. But you can change the defaults by editing the file config.auto.tfvars and change the setting for apps . Optionally, overwrite settings for ops . By default, the configuration from apps is inherited.

• name_prefix (required)

  Defaults to name_prefix = "kind" .

• base_domain (required)

  Defaults to base_domain = "infra.127.0.0.1.xip.io" .

• extra_nodes (optional)

  By default, KinD starter clusters come with a single control-plane node. You can add additional control-plane and/or worker nodes using the extra_nodes configuration attribute.

  Extra nodes are in addition to the first control-plane node. For example, to create a cluster with 3 control-plan and 3 worker nodes, specify 2 extra control-plane and 3 extra worker nodes like this: extra_nodes = "control-plane,control-plane,worker,worker,worker" .

TheGitOps flow defines the workspace to select and terraform commands to run based on how it was triggered. For the local lab we will emulate the GitOps flow using Git hooks. Lets set this up.

1. Create a local Git remote

   git init --bare .git/localremote

   git remote add origin .git/localremote

   git push -u origin master

2. Create a new branch

   git checkout -b localpipeline

3. Create the pipeline

   cat > kbst-local-pipeline << 'EOF'

   #!/usr/bin/env bash

   set -e

   workspace_path=$1

   state_path=$2

   ref_name=$3

   docker_volumes="-v ${workspace_path}:/infra -v ${state_path}:/infra/terraform.tfstate.d -v /var/run/docker.sock:/var/run/docker.sock"

   docker_env="-e TF_IN_AUTOMATION=1"

   docker_image="kubestack-starter-kind"

   #

   #

   # Build image

   docker build -t $docker_image .

   #

   #

   # Terraform init

   docker run \

   --rm \

   $docker_volumes \

   $docker_env \

   $docker_image \

   terraform init

   #

   #

   # Terraform workspace select

   case "$ref_name" in

   refs/tags/apps-deploy-*)

   terraform_workspace="apps"

   ;;

   *)

   terraform_workspace="ops"

   ;;

   esac

   docker run \

   --rm \

   $docker_volumes \

   $docker_env \

   $docker_image \

   terraform workspace select $terraform_workspace

   #

   #

   # Terraform apply

   case "$ref_name" in

   refs/heads/master)

   terraform_args="apply --auto-approve"

   ;;

   refs/tags/apps-deploy-*)

   terraform_args="apply --auto-approve"

   ;;

   *)

   terraform_args="plan"

   ;;

   esac

   docker run \

   --rm \

   $docker_volumes \

   $docker_env \

   $docker_image \

   terraform $terraform_args

   EOF

   chmod +x kbst-local-pipeline

4. Create the hook in the remote to fire the pipeline

   cat > .git/localremote/hooks/post-update << 'EOF'

   #!/usr/bin/env bash

   set -e

   ref_name=$1

   if [ ! -d "../workspace" ]; then

   git clone -l . ../workspace

   fi

   cd ../workspace

   # we're running as a hook of localremote

   # and localremote is a bare repository

   # we have to fix GIT_DIR to have workspace

   # correctly have a working dir

   export GIT_DIR=.git

   git log -1

   git fetch origin --tags --prune

   ref_origin="${ref_name/'refs/heads'/'remotes/origin'}"

   hash=$(git rev-parse --verify ${ref_origin})

   git reset --hard $hash

   workspace_path=$(realpath .)

   state_path=$(realpath ../../terraform.tfstate.d)

   exec ./kbst-local-pipeline $workspace_path $state_path $ref_name

   EOF

   chmod +x .git/localremote/hooks/post-update

5. Commit and push the pipeline

   If we now commit our changes and push them to our local remote, our newly created pipeline will be triggered for the first time. Because it's triggered from a branch it will run against the ops environment but because the branch is not master , it will only run terraform plan and not terraform apply .

   git add kbst-local-pipeline

   git commit -m "Add local pipeline"

   git push origin localpipeline

6. Merge into master branch

   Next, we merge the localpipeline branch into master and push this merge to trigger a pipeline run for the master branch. This will apply the changes to the ops environment.

   git checkout master

   git merge localpipeline

   git push origin master

7. Set a apps-deploy tag

   Finally, to apply the changes we just validated against the ops environment also against the apps environment set and push an apps-deploy tag. Triggered from a tag, the pipeline selects the apps environment and runs terraform plan .

   git tag apps-deploy-0

   git push origin apps-deploy-0

To recap, you have created a local repository holding the configuration for both an ops and an apps environment. Each has a local Kubernetes cluster powered by KinD (Kubernetes in Docker) and you can select the Terraform workspace to control which one of the environments changes are applied to.

If you also followed the optional GitOps section, you even have a local GitOps pipeline and can simulate the entireGitOps flow from creating a feature branch and getting feedback on the changes to validating the changes against ops and finally applying them against apps .

You can keep the local lab to use it as a development environment. Or optionally, tear it down using terraform destroy

1. Optional: Destroy the local lab

   # Exec into the bootstrap container again

   docker run --rm -ti \

   -v ` pwd ` :/infra \

   -v /var/run/docker.sock:/var/run/docker.sock \

   kubestack-starter-kind

   # Destroy the apps environment

   terraform workspace select apps

   terraform destroy --auto-approve

   # Destroy the ops environment

   terraform workspace select ops

   terraform destroy --auto-approve

   # Exit the bootstrap container

2. Optional: Cleanup localhost

   # to cleanup the repository, simply delete it

   rm -rf kubestack-starter-kind/

   # and cleanup the unused Docker images

   docker system prune