10

Extracting values from environment variables in tox

 4 years ago
source link: https://schinckel.net/2020/06/02/extracting-values-from-environment-variables-in-tox/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

Extracting values from environment variables in tox

Tox is a great tool for automated testing. We use it, not only to run matrix testing, but to run different types of tests in different environments, enabling us to parallelise our test runs, and get better reporting about what types of tests failed.

Recently, we started using Robot Framework for some automated UI testing. This needs to run a django server, and almost certainly wants to run against a different database. This will require our tox -e robot to drop the database if it exists, and then create it.

Because we use dj-database-url to provide our database settings, our Codeship configuration contains an environment variable set to DATABASE_URL. This contains the host, port and database name, as well as the username/password if applicable. However, we don’t have the database name (or port) directly available in their own environment variables.

Instead, I wanted to extract these out of the postgres://user:password@host:port/dbname string.

My tox environment also needed to ensure that a distinct database was used for robot:

[testenv:robot]
setenv=
  CELERY_ALWAYS_EAGER=True
  DATABASE_URL={env:DATABASE_URL}_robot
  PORT=55002
  BROWSER=headlesschrome
whitelist_externals=
  /bin/sh
commands=
  sh -c 'dropdb --if-exists $(echo {env:DATABASE_URL} | cut -d "/" -f 4)'
  sh -c 'createdb $(echo {env:DATABASE_URL} | cut -d "/" -f 4)'
  coverage run --parallel-mode --branch manage.py robot --runserver={env:PORT}

And this was working great. I’m also using the $PG_USER environment variable, which is supplied by Codeship, but that just clutters things up.

However, when merged to our main repo, which has it’s own codeship environment, tests were failing. It would complain about the database not being present when attempting to run the robot tests.

It seems that we were using a different version of postgres, and thus were using a different port.

So, how can we extract the port from the $DATABASE_URL?

commands=
  sh -c 'dropdb --if-exists \
                -p $(echo {env:DATABASE_URL} | cut -d "/" -f 3 | cut -d ":" -f 3) \
                $(echo {env:DATABASE_URL} | cut -d "/" -f 4)'

Which is all well and good, until you have a $DATABASE_URL that omits the port…

dropdb: error: missing required argument database name

Ah, that would mean the command being executed was:

$ dropdb --if-exists -p  <database-name>

Eventually, I came up with the following:

sh -c 'export PG_PORT=$(echo {env:DATABASE_URL} | cut -d "/" -f 3 | cut -d ":" -f 3); \
              dropdb --if-exists \
                     -p $\{PG_PORT:-5432} \
                     $(echo {env:DATABASE_URL} | cut -d "/" -f 4)'

Whew, that is a mouthful!

We store the extracted value in a variable PG_PORT, and then use bash variable substitution (rather than tox variable substitution) to put it in, with a default value. But because of tox variable substitution, we need to escape the curly brace to allow it to be passed through to bash: $\{PG_PORT:-5432}. Also note that you’ll need a space after this before a line continuation, because bash seems to strip leading spaces from the continued line.


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK