Snapshot Testing Going Beyond UI

Recently I gave a talk on BerlinJS community event where I wanted to share my recent experience with Snapshot testing. Pretty sure, you heard that term since it recently popularized with Jest testing framework.

Snapshot testing approach is not something new. First time I heard about it in famous “Working With Legacy Code” by M. Feathers book. The approach of “character-based” testing resonated with me, due to its simplicity and perfect fit for testing complex systems. However, that time there was no good tools that easily allowed to integrate that approach into a development process.

Later on, Llewellyn Falco released his Approval Testing framework. From my perspective, it was the first approach to bring more excellent UX into snapshot testing.

Let’s talk a bit about snapshot testing fundamentals and see, that its application goes far beyond just testing React components.

The State and Representation

For snapshot testing, it all comes to the understanding of the system’s state and the representation of the state (output of the system).

Depending on the type of the system, you have a different notion for state and representation, to make it a little bit more clear I would try to describe it for three different developers roles — frontend, backend, and mobile.

Front-end people

In the case of Redux based application, the state is Redux state. So, it’s one big object that holds the values of all UI controls and data which comes from the server. If you embrace MVC pattern for building apps, the state of your application is the combination of models which application holds.

The representation of the state is actual HTML/CSS code (produced by React, for example) which is rendered by a browser as an application UI.

Back-end people

Very typically the state of the system is the combination of databases, memory, files and other services which system has access to. So, all inputs/outputs involved in systems behavior forming its state.

The representation of the state is JSON / HTML / XML or any other format which endpoint responds with.

Mobile people

I guess it’s quite similar to front-end where the state is a model (or set of models) that application holds in memory (or local storages, key-value or SQL based).

The representation of the state is UI drawn by the mobile platform.

The Essence of Snapshot Testing

In essence, the snapshot testing is an ability to “freeze” the state and get the representation of the system (snapshot), serialized in a form convenient for computers and humans.

If the system got changed, we take the next snapshot and compare for a previous one.

If the snapshots are different, the test become “red”. We decide if the difference is “right” or “wrong” and either “approve” next snapshot or fix the problem cause a regression.

Let me explain the formula a little bit more,

f() — the function, the behavior of the system;

State — the state, data that comes as input to the system;

R — the representation of system output;

The most important thing for snapshot testing is to be able to get the representation in the form which is easy to understand (read) and compare with a previous state (diff).

Readability

Despite unit testing with asserts, where you express expected result as,

expect(result).toEqual(something);

With snapshots, you don’t know what the “something” is. However, you see how “it looks like” . Since you have to conclude is that snapshot representation right or not, the readability of it becomes crucial.

Diff-ability

In the same time, the representation format should be suitable for diffing.

So here it comes to challenge,

As the better readable format is as less it’s diff-able and otherwise.

Let’s consider a couple of examples,

1. We have an accounting reporting system, which state is the database with all invoices, expenses, customers, etc. and the output is a PDF file for a yearly report. The PDF is easy to read, but since it’s a binary format is hard to diff and moreover understand what exactly changed by reading the diff.

2. We have a web application, with a set of different components on screen, we can represent the state as actual HTML, which is perfect to diff, but our brain is not browser it’s tough to render that HTML in our minds, so even if we can read it, it’s hard to make a conclusion that it “looks” right.

So, it’s important to keep a proper balance between these two properties.

Fortunately, in practice, we are dealing with simple representations of smaller JSON’s or HTML’s which are easy to understand and compare.

Benefits of snapshot testing

1. As I mentioned above, the testing approach is more straightforward with snapshots. Yes, it’s no longer test-first, but rather test-after approach. You write the code; you test it; you derive the representation and freeze it till the next test run.
2. It suits for complex representations as big JSON and HTML outputs, so instead of hundreds of assert() calls you have one  .toMatchSnapshot() call.
3. Snapshot becomes a visualization for code changes. As part of the codebase and pull requests, you immediately see what code changes do, by reviewing the changes to snapshots.