End-to-End Testing Web Apps: The Painless Way
source link: https://www.tuicool.com/articles/hit/2UfQn2f
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Okay, I know you’re skeptical. Other guides have promised you painless web app tests only to reveal that their solution requires some hyper-specific tech stack or a paid third-party service. I won’t do that to you.
This guide provides a straightforward and flexible template for end-to-end tests that you can apply to almost any web app. There are only two requirements:
- Your app can run in Docker.
- Your app is compatible with the Chrome browser.
That’s it! You can test a Ruby app, a React app, an Enterprise Java Beans app, or even some wacky web stack you invented. And it doesn’t matter if you’re developing on Windows, Linux, or Mac. Best of all, you don’t have to perform convoluted configuration or install any software beyond Docker.
This tutorial uses free, open-source tools, and you can run them without registering an account anywhere. When it comes time to run your tests in a continuous integration environment like Circle or Travis, you don’t need to do anything special — you’ll run your tests with the same one-line command you use on your development machine.
Cypress, the star of the show
The tool that makes this testing possible is Cypress , a recent entrant to the field of browser automation. It’s an open-source end-to-end testing framework with a full-time team actively developing it. Their business model is similar to Docker’s in that both companies publish free-open source tools and fund development by selling managed services for those tools.
Cypress is an open-source tool for automated web app testing.I first discovered Cypress last year after seeing Gleb Bahmutov demonstrate it at a regional software conference. When he mentioned that Cypress had no dependencies on Selenium , I was intrigued. All my previous experience with end-to-end testing was awful, and Selenium was always at the root of my pain.
Selenium is the oldest and most prevalent browser automation tool, but it's clunky and outdated.Selenium is, by far, the most popular browser automation framework, but it also has all of the problems you’d expect of a Java-based tool designed 15 years ago. It’s a pain to install, its syntax is awkward, and it offers scant insights when your tests fail. In Gleb’s slick demos of Cypress, it promised to address all of these headaches.
One slick feature of Cypress is that it records the browser at every step of your test to help you diagnose failures.I eagerly read the Cypress docs but was disappointed to learn that virtually all of Cypress’ documentation assumed that the user had a Node.js stack and developed in a graphical environment rather than a headless console.
Still, Cypress seemed to have a promising future. A year later, I checked in on their progress and discovered a new sample application that combined Cypress with Docker Compose. Suddenly, everything clicked. Once I saw Cypress working under Docker Compose, it was clear how to adapt that pattern to any web app. Today, I’m showing you that pattern and how to use it in your apps.
A reusable pattern for end-to-end tests
Combining Cypress with Docker Compose yields a test pattern that’s flexible enough to apply to almost any web app. Unlike other testing tools that make assumptions about your app’s implementation, this solution wholly decouples your test framework from the app you’re testing.
How Docker Compose, Cypress, and the web app fit togetherDocker Compose allows you to run Cypress in one container and your app in another. Your app doesn’t need to know anything about Cypress, and the only thing Cypress needs to know about your app is the network port to send HTTP requests.
A simple web app to test
As an example web app to test, I present Sentimentalyzer: the world’s dumbest text sentiment analyzer. It tries to guess the user’s mood from a sample of their writing.
If you enter the text It's a nice day today
, Sentimentalyzer deduces that you’re happy:
If you enter the text Who ate ALL MY WAFFLES?
, Sentimentalyzer assumes that you’re angry:
The algorithm is simple: if more than 50% of the characters are uppercase, the user is yelling, so they must be mad. Otherwise, Sentimentalyzer assumes the user feels okay.
Project layout
Here’s the file layout for my example project :
main.go <- source for my web app, Sentimentalyzer Dockerfile <- defines how to run Sentimentalyzer in a Docker container e2e/ <- folder that contains all the files for my end-to-end tests cypress.json <- Cypress configuration docker-compose.yml <- glue that binds together my app container with the Cypress container integration/ spec.js <- defines the end-to-end test for Sentimentalyzer
All of the production logic is in the root folder, while all the end-to-end testing code is in the e2e
folder.
Run Sentimentalyzer locally
I’m deliberately not showing the app’s source code here to emphasize the fact that you can write Cypress tests without ever seeing the implementation of the app itself. Sentimentalyzer happens to be a Go app, but the tests would be the same had I implemented it in Python or Angular. If you’re curious, the source code is on Github .
To play with Sentimentalyzer on your machine, run the following commands:
git clone https://github.com/mtlynch/hello-world-cypress.git cd hello-world-cypress docker build --tag sentimentalyzer . docker run \ --interactive \ --tty \ --env PORT=8123 \ --publish 8123:8123 \ sentimentalyzer
The above command spawns a Sentimentalyzer server on your local machine at http://localhost:8123 .
Now that I can run my app in a Docker container, I’m ready to use Cypress to create an end-to-end test for it.
Creating an end-to-end test
To write your first Cypress end-to-end test, you only need three files:
cypress.json
This file specifies Cypress’ configuration options :
{ "pluginsFile": false, "supportFile": false }
These settings aren’t terribly interesting, but I set them to false
to prevent Cypress from auto-generating unnecessary helper files.
docker-compose.yml
This file defines a Docker container for Sentimentalyzer and a Docker container for Cypress and allows them to talk to each other:
version: '3.2' services: sentimentalyzer: build: ../ environment: - PORT=8123 cypress: image: "mtlynch/cypress:3.2.0" depends_on: - sentimentalyzer environment: - CYPRESS_baseUrl=http://sentimentalyzer:8123 working_dir: /e2e volumes: - ./:/e2e
A few lines are worth calling out:
image: "mtlynch/cypress:3.2.0"
In a bizarre gotcha , Cypress’ official Docker image doesn’t contain Cypress . mtlynch/cypress
is my custom Docker image that runs Cypress right out of the box. The 3.2.0
tag, as you might expect, contains the 3.2.0 release of Cypress.
You’re welcome to use my Cypress Docker image, fork it , or
nicely ask the Cypress team to maintain an official Docker image that includes the Cypress binary . ( Edit : Looks like the Cypress folks are considering an official image)
depends_on: - sentimentalyzer
The depends_on
stanza ensures that Sentimentalyzer is up and running before Cypress starts executing its tests.
environment: - CYPRESS_baseUrl=http://sentimentalyzer:8123
The CYPRESS_baseUrl
environment variable gives Cypress the URL where it can access Sentimentalyzer. Because Cypress and Sentimentalyzer run in the same Docker Compose configuration, Cypress can send Sentimentalyzer network requests using its container name ( sentimentalyzer
) as a hostname.
working_dir: /e2e volumes: - ./:/e2e
Lastly, I use Docker’s volume mounting feature so that the Cypress Docker container shares some of the host machine’s filesystem.
Everything in the host machine’s ./e2e
directory appears in the Docker container under the path /e2e
. This ensures that when Cypress writes logs, screenshots, or videos during its execution, they’re available immediately on the host machine without any manual copy from container to host. Binding the host volume in this way also makes it easy to edit and re-run your tests without having to rebuild your entire Docker image.
The working_dir
line ensures that Cypress treats the /e2e
directory as its current folder in the filesystem.
integration/spec.js
Now that the configuration is out of the way, it’s time for the fun part: writing tests.
it('detects angry sentiment', () => { cy.visit('/analyze') cy.get('#feelings') .type('I REALLY need some COFFEE') cy.get('form').submit() cy.get('.results p') .should('contain', 'You are feeling: Angry') }) it('detects content sentiment', () => { cy.visit('/analyze') cy.get('#feelings') .type('I think coffee in the morning is just swell!') cy.get('form').submit() cy.get('.results p') .should('contain', 'You are feeling: Content') })
Even if you’re unfamiliar with the Cypress API , its semantics are readable enough that you probably understand the tests intuitively. In plain English, both tests follow the same sequence:
/analyze
I’ll walk through the first test line by line:
cy.visit('/analyze')
This line tells Cypress to load the /analyze
path of Sentimentalyzer in the browser. Cypress combines this with the CYPRESS_baseUrl
environment variable, which I defined in, above, so the full URL is http://sentimentalyzer:8123/analyze
. You can’t access that URL from your development machine, but it’s a valid address within the Cypress container.
cy.get('#feelings') .type('I REALLY need some COFFEE')
Next, I tell Cypress to find the text field. This is easy because the text field has a unique ID, feelings
, so I specify the element using CSS selector syntax: #feelings
.
The type()
function , tells Cypress to type some text into the field I specified.
Next, Cypress has to submit the form. Cypress provides a submit()
function for this common task. There’s only one <form>
element on the page, so it’s trivial to retrieve it with the CSS selector of form
and then to submit the form:
cy.get('form').submit()
Submitting the form should bring Cypress to Sentimentalyzer’s results page. Cypress needs to check for the text "You are feeling: Angry"
but it’s a bit trickier since the <p>
tag that contains it lacks an ID attribute:
I again use CSS selector syntax to locate the relevant text by specifying a <p>
element under a DOM node whose class is "results"
:
cy.get('.results p') .should('contain', 'You are feeling: Angry')
The contain
assertion verifies that the <p>
tag contains the text I expect.
Running my tests
Now that everything is in place, it’s time to see Cypress in action. I run my tests with a simple command:
cd e2e docker-compose up --exit-code-from cypress
The --exit-code-from cypress
flag tells Docker Compose to use the Cypress container’s exit code as the exit code for the docker-compose
command. This means that the command has an exit code of zero when the tests pass and a non-zero exit code on test failure. This behavior is handy for build scripts or continuous integration configurations that use a command’s exit code to determine if it succeeded.
Here’s what the whole process looks like from the console:
Cypress creates a video recording of every test run. This is my favorite feature, as it’s a tremendous help in diagnosing test failures:
Your browser does not support the video tag.Test failure screenshots
Above, I showed you a test that passed. What happens when a Cypress test fails? It still generates a video of the test run, but it also outputs a screenshot showing the assertion that failed:
Screenshot that Cypress generated when my test failed (Cypress expected the word “Furious” but instead found “Angry”)This solves a major pain point I experienced with other tools. Selenium supports screenshots but only before or after an assertion. That limitation led to frustrating scenarios where Selenium claimed the test failed, but the screenshot showed correct behavior because the browser state changed after the test failed.
Cypress avoids this problem because its screenshots happen concurrently with its assertions. If a test fails, the screenshot shows you precisely what Cypress saw at the time of the failure.
Adapting this for your web app
Those three files are all you need to start end-to-end testing your web app. Here are the steps:
- Copy the
e2e
folder into your project. - Replace the
sentimentalyzer
section inwith a Docker container for your app.
Source code and additional examples
The full source for this demo is available on Github:
I also created several branches to demonstrate other common Cypress scenarios:
Note: Cypress currently supports only Chrome and Electron as browser options, but cross-browser testing is one of the dev team’s top priorities .
Further reading
This guide provided a basic introduction to Cypress. For more advanced functionality, check out the official Cypress docs:
Illustrations by Loraine Yow . Thanks to Gleb Bahmutov from the Cypress team for providing early feedback on this article.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK