Integration tests are invaluable for verifying that components interact with each other appropriately in critical scenarios. At Flexport, we use integration tests regularly to check interactions between different components in our system. We ensure that consumed Kinesis events update database records appropriately, that incoming gRPC requests properly fetch and update database records, and that client GraphQL requests return and mutate data as expected.

In 2019, we started writing some of our backend services in Java using the Spring Boot framework. We’ve learned a lot about integration testing in Spring Boot since then and would like to share what has and hasn’t worked for us.

Flexport’s integration test setup

At Flexport we have three environments in which we run our integration tests: locally during development, during continuous integration (CI), and during continuous deployment (CD). We use the Bazel build system to build our Java code and run our JUnit tests. We’ve had to do a little work to get Bazel and JUnit5 to connect, but this setup is working well for us now.

When developing a feature, we write integration tests to check its correctness and run those tests locally using

bazel test //path/to/junit/integration/test/target:test

Separately, in our CI and CD pipelines, we run our full test suite. By default Bazel parallelizes different JUnit test targets, creating concurrent tests across our full suite.

bazel test //...

As our test suite grew bigger and we were writing more tests being run in parallel while touching the same database instance, we started to run into various issues. Our integration tests weren’t properly exposing bugs in our application code, and the tests themselves were flaky due to different race conditions.

With this context, here are some dos and don’ts for writing Java integration tests in a parallelizable way.

Do dynamically generate identifiers and fields with unique constraints

When writing integration tests, our entities often refer to identifiers for entities stored in other network-isolated services. When interacting with entities via external identifiers and with other fields that have uniqueness constraints, it’s important to dynamically generate these identifiers and fields so tests running in parallel won’t cause race conditions. We use the java.util.UUID library to achieve uniqueness.

For example in our consolidations planning service, we often refer to ports that are owned by an external service by identifier. To create an entity that references a port in an integration test, we use something like the following.

Do not check the absolute state of the database in integration tests

Across all languages and frameworks, this is key to a well running parallelized integration test suite running against a single database. Avoid checking the number of entities returned by a query. Avoid hard coding identifiers. For example:

Writing your integration tests like this will lead to flaky tests. If two integration tests running at the same time are adding or deleting cargo records from cargoRepository, that above assertion will fail sporadically.

Do not wrap JUnit test functions in an @Transactional annotation

In many integration test frameworks it is common to wrap each test case in a database transaction. Then, when the test completes, issuing a transaction rollback will remove any records created during the test run. At Flexport the majority of our code is written in Ruby, and wrapping each test case in a transaction works well for cleaning up db records, but it makes testing transactional behavior more difficult.

Unfortunately the Spring framework’s @Transactional tag has some undesirable consequences when used with JUnit integration tests.

1. Entities are saved automatically at the end of an @Transactional method
2. Using @Transactional hides LazyInitializationExceptions

Henrick Kakutalua’s post provides a great explanation of these issues. In summary, using @Transactional with JUnit tests can lead to false positive test runs, which erode developer confidence in the test suite’s effectiveness.

Alternatively, we can refrain from wrapping tests in transactions with @Transactional and can allow database records to accumulate during test runs. If proper integration test hygiene is kept, records from one test shouldn’t affect any other concurrent test.

Lastly, testing application transaction behavior is easier without these tags present. Using TransactionTemplate, you can set up test db state in one transaction, commit, and execute the integration test — this approach mimics production code paths executing against resting db state.

Do not truncate application database tables between test runs.

An example of this approach:

This option deletes all rows from every db table between test runs, but you will quickly run into issues when running test targets in parallel. Since these truncate operations lock each database table, deadlock issues can occur. Here’s an error we started to see periodically when trying this technique.

Do delete records created during tests between test suite runs

During a test suite execution, let the created database records sit around in the database. As long as you are dynamically generating identifiers and unique fields, records from previous tests should not affect the current test.

As your test suite becomes large, consider recreating the database at the beginning of each test suite run, or cleaning the database at the beginning of each test suite execution.

Happy testing! — Paul Scheid