Deployment Automation via SSH With Python Fabric: How It Works

April 17th 2021 new story

5

One of the most basic ways in which a project gets deployed is by SSHing into a remote host followed by executing a few basic commands. Apart from deployment, this can also be useful for running any command you want on a remote host, such as when a CI/CD pipeline is triggered. In this article I'll be taking a look on how to deploy a basic project to a remote server through Gitlab CI using Python Fabric.

0 reactions

What is Fabric?

Fabric is an open source python library that is used to execute commands remotely over SSH.

0 reactions

Fabric is compatible with Python 2 and 3. To use it with Python3, install it with:

0 reactions

pip3 install fabric

Since the end goal here is to use fabric in an automated deployment pipeline, you don't actually need to have fabric installed on your local machine (unless that is where your pipeline runs).

0 reactions

Installing fabric also installs the

fab

0 reactions

Creating a fabfile

Create a file named

fabfile.py

0 reactions

from fabric import task


@task
def deploy(ctx):
    print("Inside the task!")

This defines a task called "deploy" which can be passed as an argument to the fab binary for execution. The

@task

deploy 

ctx

0 reactions

On executing this by running

fab deploy

fabfile.py

0 reactions

Now to use this to do what Fabric is intended for - connecting a remote host and executing commands in it.

0 reactions

from fabric import Connection, task


@task
def deploy(ctx):
    with Connection("HOST") as c:
        with c.cd("/home/project/path/"):
            c.run("docker-compose down")
            c.run("git pull origin master --recurse-submodules --rebase")
            c.run("docker-compose up --build -d")

Here, replace

HOST

c.cd()

c.run()

0 reactions

In this example I'm making use of Python context managers with the help of

with

0 reactions

By now you might have noticed just the host name is often not enough to establish a connection. What about the username and the private key to establish an SSH connection? This isn't immediately obvious, but because Fabric uses a library called Paramiko under the hood for handling the SSH side of things, we can pass arguments to Paramiko's

SSHclient.connect

connect_kwargs

0 reactions

from fabric import Connection, task


@task
def deploy(ctx):
    with Connection(
        "HOST",
        user="USERNAME",
        connect_kwargs={"key_filename": "~/.ssh/your_key"}
    ) as c:
        with c.cd("/home/project/path/"):
            c.run("docker-compose down")
            c.run("git pull origin master --recurse-submodules --rebase")
            c.run("docker-compose up --build -d")

Paramiko's

SSHclient.connect

key_filename

0 reactions

You can also pass the SSH key as an instance of Paramiko's

pkey.Pkey

0 reactions

Of course, it's much safer to load all of these as environment variables. When using Gitlab CI, while setting the environment variables you can choose to make the key available as a file instead of a string, which is needed to make it work with the

key_filename

0 reactions

Here is a screenshot of the Variables section under Gitlab CI/CD settings with the hostname added as a string and the key added as a file.

0 reactions

You might want to load these values in other ways depending on how your team is organized and as per your security requirements.

0 reactions

With these environment variables in place, the fabfile changes to this:

0 reactions

import os
from fabric import Connection, task


@task
def deploy(ctx):
    with Connection(
        os.environ["HOST"],
        user="USERNAME",
        connect_kwargs={"key_filename": os.environ["DEPLOY_KEY_FILE"]},
    ) as c:
        with c.cd("/home/project/path/"):
            c.run("docker-compose down")
            c.run("git pull origin master --recurse-submodules --rebase")
            c.run("docker-compose up --build -d")

Setting up Gitlab CI

0 reactions

Now that the fabfile is ready, executing it with the Python Fabric package is all we have to do. This can be done in any way you want - manually, using Github actions, etc. Here I have an example of running it with Gitlab's CI.

0 reactions

To do this, create a

.gitlab-ci.yml

0 reactions

image: "python:3.6"

stages:
  - deploy

deploy_to_production:
  stage: deploy
  script:
    - pip3 install fabric
    - fab deploy
  only:
    - master

This is a basic configuration file that should be easy to understand if you're familiar with Gitlab's CI. It just has one job called

deploy_to_production

pip3 install fabric

fab deploy

fabfile.py

0 reactions

Conclusion

0 reactions

Fabric has quite a lot more options than what is outlined here - such as dealing with sending the sudo password to the remote host when it's required, dealing with multiple hosts etc. It's worth going through their documentation if you're planning to make use of it.

0 reactions

In conclusion, this is a really simple and effective way to programmatically build scripts to be executed over SSH and trigger them from common methods such as through a CI/CD tool.

0 reactions

I hope you found this post useful. You can find me on Twitter and LinkedIn.

0 reactions

5