Adopting CI/CD in Your Java Project with the Gitflow Branching Model

A thousand years ago I was at a technology conference, where I came upon a newfangled gizmo on exhibit called “Git”. I learned that it was a next generation source control tool, and my initial reaction was - why do we need that, we already have SVN? That was then. Today development teams are moving to Git in droves, and a huge ecosystem has evolved around middleware and plugins.

Gitflow is a collaborative branching model that exploits the power, speed, and simplicity of Git branching. Introduced by Vincent Driessen in his classic 2010 blog “ A Successful Git Branching Model ”, Gitflow takes the pain out of collaborative development by allowing teams to isolate new development from completed work, allowing you to cherry-pick features for release, while still encouraging frequent commits and automated testing. As a by-product it produces cleaner code, by promoting code-reviews, even self-code reviews, thereby exposing bugs, opportunities for refactoring, and optimizations.

But when it comes to implementing Gitflow in a CI/CD environment, the particulars are very specific to your development environment, and there are countless possibilities. Consequently the documentation is sparse; given the well-known branch names - master, develop, feature , etc., which branches do we build, which do we test, which do we deploy snapshots, which deploy releases, and how do we automate deployments to Dev, UAT, Prod, etc.?

These are frequently asked questions at conferences where we have presented, and in this article we would like to share a solution we have developed in our own work at a large financial technology firm.

The project described here uses Java and Maven, but any environment could be similarly adapted. We use GitLab CI, but Jenkins or a GitHub CI plugin could also be used; we use Jira for issue tracking, IntelliJ IDEA as our IDE, Nexus as our dependency repository, and we use Ansible for our automated deployment, but any similar tools can be substituted.

First, let’s see how we got here from there.

Evolution

In antediluvian times, developers would spend weeks or months building an application feature, whence they would hand off the “completed” work to an “integrator” -- a well-meaning and dedicated human chap, who would take all such features, integrate them, resolve conflicts, and prepare them for release. The integration process was a daunting, error-fraught endeavor, unpredictable in schedule and consequence, breeding the well-deserved appellation “integration hell”. Then around the turn of the century Kent Beck released his seminal book “Extreme Programming Explained”, which advocated the concept of “ continuous integration ”; the practice of building and integrating code in an automated fashion, every few hours. Not long after that, Martin Fowler’s Thoughtworks open-sourced Cruise Control, one of history’s first CI automation tools.

Enter Gitflow

Gitflow advocates the use of feature branches for developing individual features, and separate branches for integration and release, as we will see. The following graphic reprinted from Vincent Driessen’s blog is now very familiar to Git development teams.

As Git users, we are all familiar with the branch called “master”; this is the “trunk” branch created by Git by default when we first initialize any Git project. Before adopting Gitflow, you’ve most likely been committing to your master branch.

Kicking Off Gitflow

To get a project started with Gitflow, there is a one-time initialization step where you create a branch off master called “ develop ”. From then on, develop becomes the catch-all branch, where all of your code is deposited and tested, essentially becoming your primary “integration” branch.

As a developer, you will never commit directly to the develop branch, and you will never, never commit directly to master . Master is referred to as a “stable” branch - containing only work that is production ready, either released or ready for release. If it’s in master, it is a past or future production release.

The develop branch is referred to as “unstable”; perhaps a bit of a misnomer - it is stable in that it contains code that is destined for release; and it must compile and tests must pass; just that it contains work that may or may not be complete, and therefore “unstable”.

So where do we do our work? That’s where the rest of the picture starts to materialize:

You get a new Jira to work on. Immediately you branch a feature branch, typically from develop if that is at a stable point, or else from master :

We have agreed on the convention of naming our feature branches as “feat-” followed by the Jira issue number. (If there are more than one Jira issue, just use the Epic or Parent task, or one of the main issue numbers, followed by a very brief description of the feature.) For example “feat-SDLC-123-add-name-field”. The “feat-” prefix provides a pattern that the CI server can use to identify this as a feature branch. We will see why this is important in a few paragraphs. In this example, SDLC-123 is our Jira issue number, which gives us a visual link to the driving issue, and the remaining description gives us a brief narrative describing the feature.

Development now proceeds in parallel, everyone working on their feature branches concurrently, some teams working on the same branch completing that feature, others working on different features.

Releases, Snapshots, and Shared Repositories

Let’s spend a few words to clarify this. In most enterprises, there is a single dependency repo such as Sonatype Nexus. This repo contains two kinds of binaries. “SNAPSHOT” binaries are usually named using a semver (three-part dot-delimited) version, followed by the word “-SNAPSHOT”, (e.g. 1.2.0-SNAPSHOT). Release binaries are versioned with the same name except without the “-SNAPSHOT” suffix (e.g. 1.2.0). Snapshot builds are unique in that any time you build a binary with that snapshot version, it replaces any previous binary that had that same name. Release builds are not that way; once you build a release build you can take it to the bank that the binary associated with that version will never be changed in Nexus.

Now, imagine you are working on feature X and your companion team is working on feature Y. You both branched off develop at the same time, so you both have the same base version in your POM (say 1.2.0-SNAPSHOT). Now let’s say you run your build and deployed your feature branch to Nexus, and soon after that, your companion team ran their build and deployed that to Nexus. In such a scenario you would never know which feature binary was in Nexus, since 1.2.0-SNAPSHOT would refer to two different binaries corresponding to the two distinct feature branches (or more if there are more such feature branches!) This is a very commonly occurring conflict.

GitLab CI

Yet we instruct teams to commit often and early! So how do we avoid such conflicts? The answer is to tell GitLab CI to build, but just not deploy to Nexus, by associating the “feat-” branches with a Maven verify lifecycle step (which builds locally and runs all tests), rather than a Maven deploy step (which would send the snapshot binary to Nexus).

GitLab CI is configured by defining a file (called .gitlab-ci.yml) in the project root, which contains the exact CI/CD execution steps. The beauty of this feature is that the run script is then associated with your commit, so you can vary it based on a commit or a branch.

We have configured GitLab CI with the following job, containing regex and script for building feature branches:

feature-build:
 stage:
   build
 script:
   - mvn clean verify sonar:sonar
 only:
   - /^feat-\w+$/

Teams are encouraged (nay, mandated!) to commit frequently. This runs your tests in isolation, ensuring that your feature work has not broken anything, and allowing you to add tests to the code you have changed.

Coverage Driven Development

Now would be a good time to discuss test coverage. IntelliJ idea has a “coverage” run-mode, that allows you to run your test code with coverage, (either in debug or run mode), and paints your margins green or pink, depending on whether that code was covered or not. You can (and should) also add a coverage plugin (such as Jacoco) to Maven, so that you can receive coverage reports as part of your integration build. If you are using an IDE that does not color the margins, you can pull these reports to locate swaths of uncovered code.

[One sidebar note on this - unfortunately there are still many professional development teams who although pontificating an intractable orthodoxy on automation and development, yet for one reason or another have been remiss in amplifying their test coverage. Now, we won’t be the ones to tell such teams to go back and add tests to every uncovered piece of code; but as good developer citizens, we consider it our duty to introduce tests at least for the code we have added or modified. By reviewing the coverage-color coded margins against the code we introduced, we can quickly identify opportunities for introducing new tests.]

Tests are executed as part of the Maven build. Maven test phase executes unit tests (designated by a name that starts with Test-something.java or ends with Test.java, Tests.java, or TestCase.java). Maven verify phase (requires Maven Failsafe plugin) also executes integration tests. A call to mvn verify triggers the build, followed by a pipeline of lifecycle phases including test and verify. We also recommend installing the SonarQube, and the Maven SonarQube plugin, for static code analysis during the test phase. In our model, every branch commit or merge executes all of these tests.

Continuous Integration

Back to Gitflow. We have now done some more work on our feature, we have been committing to our feature branch, but in the spirit of continuous “integration” we want to ensure it plays nicely with all of the other team feature commits. So by policy we agree that all development teams merge to the develop branch at least once per day.

We also have a policy that we enforce inside of GitLab, that we cannot merge into develop without a code review, in the form of a merge request:

Depending on your SDLC policy you can force developers to do a code review with someone else by endowing your merges with a list of approvers. Or you can enforce a more relaxed strategy by allowing developers to perform their own code reviews after viewing their own merge request. This strategy works nicely in that it encourages developers to at least review their own code, but like any honor system, it comes with obvious risks. Note that since the binary will never be deployed to Nexus or otherwise shared, the POM version contained in the develop branch is irrelevant. You can call it 0.0.0-SNAPSHOT, or just leave the original POM version from whence it was branched.

Finally, after some number of days, the feature is complete, it is fully merged into develop and declared stable, and there are a number of such features that are ready for release. Remember that at this point, we have run our verification tests on every commit, but we have not yet deployed so much as a SNAPSHOT to Nexus. That is then our next step.

At this point we branch a release branch off develop. But in a slight departure from traditional Gitflow, we don’t call it release; rather we name the branch after the release version number. In our case, we use a 3 part semantic versioning, so if it is a major release (new features or breaking changes) we increment the major (first) number, a minor release we increase the minor (second) number, and if a patch, the third. So if the previous release was 1.2.0 the coming release might be 1.2.1 and the snapshot pom version would then be 1.2.1-SNAPSHOT. So our branch would be named accordingly 1.2.1.

Configuring the Pipeline

We have configured our GitLab CI pipeline to recognize that a release branch has been made (a release branch is identified by a semver three part dot delimited number; in regex-speak: \d+\.\d+\.\d+). The CI/CD runner is configured to extract the release name from the branch name, and to use the version plugin to change the POM version to include the SNAPSHOT corresponding to this branch name (1.2.1-SNAPSHOT in our example).

release-build:
 stage:
   build
 script:
   - mvn versions:set -DnewVersion=${CI_COMMIT_REF_NAME}-SNAPSHOT
   # now commit the version to the release branch
   - git add .
   - git commit -m "create snapshot [ci skip]"
   - git push
   # Deploy the binary to Nexus:
   - mvn deploy
 only:
   - /^\d+\.\d+\.\d+$/
 except:
   - tags

Notice the [ci skip] in the commit message. This is critical for preventing looping, where each commit would trigger a new run and a new commit!

After the CI runner makes the change to the POM, the runner commits and pushes the updated pom.xml (now containing a version that matches the branch name.) Now, the remote release branch POM contains the correct SNAPSHOT version for that branch.

GitLab CI, still identifying this release branch by the semantic versioning pattern ( /^\d+\.\d+\.\d+$/ , for example 1.2.1) of its name, recognizes that a push event has occurred on the branch. The GitLab runner executes mvn deploy to produce a SNAPSHOT build and a deploy to Nexus. Ansible now deploys that to a dev server, where it is available for testing. This step executes for all pushes to the release branch. This way, small tweaks that the developers make to the release candidate trigger a SNAPSHOT build, SNAPSHOT release to Nexus, and deployment of that SNAPSHOT artifact to the development servers.

We have omitted the Ansible deployment scripts because those are very personal to your specific development and release design, and can perform whatever actions are needed for artifact deployment, including restarting services after new artifacts are installed, updating cron schedules, changing application configuration files, etc.

And at long last we merge to master , triggering Git to tag the release with the semver release number from the source release branch name, deploy the whole wad to Nexus, and run our sonar tests.

Note that in GitLab CI, any things you want to have hang around for the next job step, need to be designated as artifacts. In this case we are going to deploy our jar artifact using Ansible, so we designate that as a GitLab CI artifact.

Here's the code:

master-branch-build:
  stage:
	build
  script:
	# Remove the -SNAPSHOT from the POM version
	- mvn versions:set -DremoveSnapshot
	# use the Maven help plugin to determine the version. 
    # Note the grep -v at the end, to prune out unwanted log lines.
	- export FINAL_VERSION=$(mvn --non-recursive help:evaluate 
      -Dexpression=project.version | grep -v '\[.*')
	# Stage and commit the binaries (again using [ci skip] 
    # in the comment to avoid cycles)
	- git add .
	- git commit -m "Create release version [ci skip]"
	# Tag the release
	- git tag -a ${FINAL_VERSION} -m "Create release version"
    # and push it back to Gitlab
	- git push 
	- mvn sonar:sonar deploy
  artifacts:
	paths:
	# list our binaries here for Ansible deployment in 
    # the master-branch-deploy stage
  	- target/my-binaries-*.jar
  only:
	- master
 
master-branch-deploy:
  stage:
	deploy
  dependencies:
	- master-branch-build
  script:
   # You could deploy artifacts (target/my-binaries-*.jar)
   # here, using Ansible
  only:
	- master

Squashing Bugs

During testing, bug fixes might be discovered. These are fixed right on the release branch, and then merged back to develop . (Develop always contains everything that was or will be released.)

Finally, the release branch is approved, and it is merged into master. Master has an enforced GitLab policy never to accept merges except from a release branch. The GitLab runner checks out the merged code into master, which still has the release branch SNAPSHOT version. The GitLab runner uses the Maven versions plugin again to execute the versions:set goal with the removeSnapshot parameter set. This goal will remove the “-SNAPSHOT” from the POM version, and the GitLab runner will push this change to the remote master , tag the release, increment the POM version to the next SNAPSHOT version, and deploy that to Nexus. That is deployed to UAT for QA and UAT testing. Once the artifact is approved for release to production, production services teams will take the release artifact and deploy it to production. (This step can be automated as well via Ansible, depending on your corporate policy.)

Patches and Hotfixes

There is one more side workflow we must mention, and that is for patches or hotfixes. These are triggered when an issue is found in production or during testing of a release artifact, for example a bug or performance issue. Hotfixes are similar to release branches. They are named for the release, just like a release branch. The only difference is that they are branched not from develop but from master.

Work is done to complete the hotfix. Just like a release branch, the hotfix triggers a Nexus SNAPSHOT deploy, and a deploy to UAT. Once this is certified, it is merged back into develop, then it is merged into master in preparation for its release. Master will trigger the release build and deploy the release binary to Nexus.

Summary

We can summarize all of this in the following grid:

So there we have our flavor of Gitflow in all of its glory. We encourage development teams of any size to use this strategy and enjoy the following merits:

• Features are isolated – it is therefore easy to manage your own feature changes in isolation
• Feature segregation, which lets you decide which features to include in a release.
• Promotes continuous integration and frequent commits
• Promotes clean coding
• Automated testing, deployment and release to all environments.

More information

For a more traditional treatment of Gitflow using Atlassian Bamboo and BitBucket, see this . 

There is also a nifty Gitflow Maven plugin , actively maintained by Alex Mashchenko, that works much like the Maven release plugin on Gitflow steroids. This could be adapted to accommodate our proposed Gitflow implementation.

About the Authors

Victor Grazi works at Nomura Securities on enterprise infrastructure application development. An Oracle Java Champion, Victor has also served as lead editor on the Java queue at InfoQ, and has served as a member of the Java Community Process Execute Committee.

Bryan Gardner is a recent graduate of Stevens Institute of Technology, where he obtained his BS and MS in computer science. Bryan currently works at Nomura, as a software engineer on the Infrastructure Development team. He primarily spends his day working on Spring Boot backend services or working on big data pipelines with Apache Spark.