Cypress is an open-source tool for testing web applications end-to-end. I first saw Gleb Bahmutov demo Cypress at a 2018 web dev meetup in New York, and I was blown away.

I’ve been using Cypress since I saw it demoed at a dev meetup in 2018.

Before discovering Cypress, I had begrudgingly used Selenium. Cypress was a refreshing leap forward, as it offered elegant solutions to tons of pain points that made Selenium impractical to use.

I recently tried Playwright, Microsoft’s answer to Cypress. After experimenting with it for a day, I’m ready to completely switch over from Cypress to Playwright.

It pains me to say it because I have a soft spot for Cypress’ small, scrappy team. I’m certainly not enthusiastic about adding a dependency on a huge megacorp like Microsoft, but Playwright is just so much better that I can’t justify sticking with Cypress.

What follows are my notes on switching from Cypress to Playwright while they’re still fresh in my head.

Contents 🔗︎

My prior experience with Cypress and Playwright 🔗︎

I’ve written Cypress end-to-end tests for almost every web app I’ve built in the last four years. I’d rate myself as an intermediate Cypress user. Most of my Cypress needs are straightforward and only exercise the basic APIs. I’ve never written any custom plugins, but I’ve used a few third-party ones.

I’ve used Playwright for only one day. To get my hands dirty, I tried porting a test suite of one of my apps from Cypress to Playwright. I chose PicoShare, my minimalist file-sharing tool, which has just 10 end-to-end tests. I was able to port them all from Cypress to Playwright in about five dev hours, including the time it took to learn Playwright’s APIs.

I’ve never paid money for Cypress or Playwright, so I’m entitled to anything from either tool. Cypress has a paid SaaS component, but I’ve never purchased it, as it doesn’t fit into my workflow. I would have happily sponsored Cypress, as I do other open-source projects I use, but Cypress doesn’t offer any sponsorship options.

What I like about Playwright 🔗︎

Playwright is significantly faster than Cypress 🔗︎

My Playwright test suite runs 34% faster than the equivalent Cypress tests on CircleCI. On my local dev machine, Playwright gives a 5x speedup over Cypress. This is not a rigorous measurement, but it’s clear there’s a substantial speed difference between the two.

Task	Cypress	Playwright	Difference
Run tests on CircleCI	127s	84s	-34%
Run tests from development machine	40s	7s	-83%

Part of the performance difference on CI is that the Playwright Docker container is significantly smaller than the Cypress container. For local development, it’s not a big deal because you download it once, and you’re done. But when I run Cypress in CI, I have to wait for CircleCI to download and decompress a ~1 GB image each time.

cypress/included:10.9.0	playwright:v1.26.0-focal-amd64
Size	940 MB	651 MB

Playwright exposes a consistent set of assertions 🔗︎

Cypress bundles nine different third-party libraries into its tool, which creates a mishmash of inconsistent APIs. There’s should, expect, and assert, and you use a different keyword depending on the context you’re in.

For example, the following two code snippets perform identical assertions:

cy.get("#error-message").should("be.visible");

cy.get("#error-message").should(($el) => expect($el).to.be.visible);

With Playwright, there’s a single, consistent API. Asserting that the element with an ID of error-message is visible on the screen requires a simple function call:

expect(page.locator("#error-message")).toBeVisible();

Playwright does not depend on a GUI environment 🔗︎

One of Cypress’ most touted features is their desktop GUI app:

Cypress uses a desktop app to show test execution

The Cypress desktop app lets you “time travel” through your tests, so you can see what the browser window looked like at each point in your test.

But what if you develop without a GUI? I do all of my development on headless server VMs. In four years of Cypress, I’ve never used their desktop app. Instead, I run Cypress within a Docker container, which is sometimes an obstacle for a tool that expects you to work in their desktop GUI.

The GUI problem crops up again when you try to run your Cypress tests in a CI environment. There’s generally not a desktop GUI there, either. Cypress’ answer is to use their paid CI service, which is the primary way they fund the company.

I support companies monetizing their open-source product however they want, but Cypress’ CI product has never appealed to me. I want to be able to reproduce my CI environment locally with Docker containers. Playwright lets me do that, but Cypress’ CI service doesn’t.

To run Cypress on CircleCI, I had to do a bit of juggling with Docker Compose. It’s not an egregious amount of overhead, but it makes the testing stack a little more complicated than I’d like.

When I tried Playwright, it was such a breath of fresh air to use a tool that’s designed to run headless. I don’t have to do anything tricky to run Playwright in CI because it just works out of the box in a headless environment.

Playwright has the same time-travel feature as Cypress, but they implement it in a web UI instead of a desktop GUI, so it works in more environments.

Time traveling is pretty nice! Playwright’s snapshots aren’t even just static screenshots of your app. You can interact with the browser in each stage of the test, which feels a bit like magic.

Playwright has fewer feature gaps 🔗︎

Cypress makes it easy to get up and running with basic end-to-end tests, but I’ve found that as my apps grow, I frequently run into feature gaps in my testing tool.

For example, I’d add a file upload feature and then realize that Cypress can’t exercise file upload functionality. I’d stop what I’m doing and go find a third-party Cypress plugin to fill the gap.

As I was writing this, I discovered that Cypress added native support for file uploads earlier this year, but it’s a bit of a headscratcher that it took them seven years to support an extremely common scenario.

Similarly, if you want to simulate mouse hovering, a feature present in almost every web UI framework, Cypress can’t do it. That bug has been open for almost eight years.

I’m sure Playwright has its own feature gaps, but I didn’t hit any in my day of porting tests from Cypress to Playwright. All of the workarounds in my test suite for Cypress gaps had native solutions in Playwright.

Playwright requires less domain-specific knowledge 🔗︎

Back when I discovered Cypress, one of the things that appealed to me was that it was designed for JavaScript, whereas Selenium was Java-first.

For basic testing, Cypress’ semantics feel natural and familiar to someone who understands JavaScript. But when you stray off the beaten path, Cypress suddenly feels less like JavaScript and more like its own domain-specific framework.

As an example, there’s functionality in my app PicoShare to generate URLs for files that you want to share with unauthenticated users. To test the functionality, I needed to navigate through PicoShare’s sharing feature, log out of the user session, and then verify that the browser can still access the URL it generated a few steps earlier.

Here’s how I originally implemented that test in Cypress:

// Save the route to the guest link URL so that we can return to it later.
cy.get('.table td[test-data-id="guest-link-label"] a')
  .invoke("attr", "href")
  .then(($href) => {
    // Log out.
    cy.get("#navbar-log-out").click();
    cy.location("pathname").should("eq", "/");

    // Make sure we can still access the guest link after logging out.
    cy.visit($href);

    // Continue with the test
  });

You see then, so you might assume that invoke returned a Promise. But if you try to await that promise, it returns undefined because Cypress actually returned something only pretending to be a Promise.

This may not seem like a big deal, but if you ever need to refer to a value in your app dynamically, Cypress forces you into a new nested closure level for every value you need. There’s a widely supported feature request to support await, but there’s been no progress in four years, and Cypress recently stated that they currently have no plans to implement it.

Here’s what the same test looks like in Playwright:

// Save the route to the guest link URL so that we can return to it later.
const guestLinkRouteValue = await page
  .locator('.table td[test-data-id="guest-link-label"] a')
  .getAttribute("href");
expect(guestLinkRouteValue).not.toBeNull();
const guestLinkRoute = String(guestLinkRouteValue);

// Log out.
await page.locator("#navbar-log-out").click();
await expect(page).toHaveURL("/");

// Make sure we can still access the guest link after logging out.
await page.goto(guestLinkRoute);

// Continue with the test.

In Playwright, when we have a reference to a DOM element, we can call normal APIs on it like getAttribute, and we get back simple values we expect without bothering with the complexity of closures. And the promise-looking values that Playwright returns really are Promises that you can await, so the code is tidier.

Text comparisons are easier in Playwright 🔗︎

One aspect of Cypress that’s always frustrated me is how difficult it is to assert that an element contains a particular text value.

Here’s an example <p> element from PicoShare:

<p data-test-id="github-instructions">
  Visit our
  <a href="https://github.com/mtlynch/picoshare">Github repo</a> to create your
  own PicoShare server.
</p>

Unexpected text comparison results in Cypress 🔗︎

Here’s the naïve approach to asserting the text value in Cypress:

cy.get("[data-test-id='github-instructions']").should(
  "have.text",
  "Visit our Github repo to create your own PicoShare server."
);

Unfortunately, this test will fail:

Timed out retrying after 10000ms
+ expected - actual

-'\n      Visit our\n      Github repo to create\n      your own PicoShare server.\n    '
+'Visit our Github repo to create your own PicoShare server.'

Cypress is grabbing the textContent property, which includes all the whitespace around the text as it appears in the raw HTML instead of how the text appears in the browser.

You can work around this by grabbing the element’s innerText, but the syntax is convoluted and difficult to remember because it uses a totally different set of assertion APIs:

cy.get("[data-test-id='github-instructions']").should(($el) => {
  expect($el.get(0).innerText).to.eq(
    "Visit our Github repo to create your own PicoShare server."
  );
});

Unsurprising text comparisons in Playwright 🔗︎

In Playwright, the naïve assertion yields the correct behavior:

await expect(page.locator("data-test-id=github-instructions")).toHaveText(
  "Visit our Github repo to create your own PicoShare server."
);

Playwright also looks at the textContent of the element, but it automatically trims and collapses whitespace like a browser does.

You can force Playwright to look at innerText instead with a much simpler syntax than what’s available in Cypress:

await expect(page.locator("data-test-id=github-instructions")).toHaveText(
  "Visit our Github repo to create your own PicoShare server.",
  { useInnerText: true }
);

Playwright loses a few points for having two seemingly identical APIs with similar names:

• toHaveText: “Ensures the Locator points to an element with the given text. You can use regular expressions for the value as well.”
• toContainText: “Ensures the Locator points to an element that contains the given text. You can use regular expressions for the value as well.”

One API is for asserting that an element exists “with the given text” whereas the other asserts an element exists “that contains the given text?” What’s the difference between “having” text and “containing” text?

Reading more of the documentation, the difference seems to comes down to subtle differences in what you expect about an element’s child elements, but the documentation could definitely be improved.

Playwright makes it easier to navigate the shadow DOM 🔗︎

I write a lot of web apps using HTML custom elements, so my code often contains nested shadow DOMs.

In Cypress, specifying page elements within a shadow DOM is a bit awkward because you have to interrupt the CSS selector every time you encounter a shadow DOM boundary:

cy.get("#upload-result upload-links")
  .shadow()
  .find("#verbose-link-box")
  .shadow()
  .find("#link")
  .should("be.visible");

Playwright pierces the shadow DOM by default, resulting in concise CSS selectors:

await expect(
  page.locator("#upload-result upload-links #verbose-link-box #link")
).toBeVisible();

Playwright launches your app for you 🔗︎

One of Cypress’ odd design decisions is that they refuse to launch your app for you. You have to figure out how to launch your app yourself and then orchestrate your Cypress tests to start after your app is serving.

Playwright eliminates the orchestration headache and offers a simple config option to launch your app. Here’s what it looks like for PicoShare:

webServer: {
  command: "PS_SHARED_SECRET=dummypass PORT=6001 ./bin/picoshare",
  port: 6001,
},

Playwright’s logging actually works 🔗︎

One of the big pain points of Cypress is that you have to learn to live without debug logging to the terminal. Cypress has no official way to print to stdout or stderr.

If I stick in a call to console.log, nothing happens:

console.log("hello from Cypress"); // this does nothing

Cypress has its own cy.log API, so what if I try that instead?

cy.log("hello from Cypress"); // this prints nothing to the terminal

Nope, that doesn’t work either. That only prints output within the Cypress desktop GUI or Cypress’ proprietary SaaS dashboard.

Cypress developer Zach Bloomquist published an unofficial plugin for printing browser console output to the terminal, but it’s a third-party plugin and not something Cypress officially supports.

In Playwright, console.log just works: no fuss, no muss:

console.log("hello from Playwright");

When I run the test, I see the log message in the terminal output:

[chromium] › auth.spec.ts:3:1 › logs in and logs out
hello from Playwright

Playwright’s team doesn’t feel resource-constrained 🔗︎

The core Cypress repo has 2,782 open bugs, some for important feature requests that have been neglected for years. Sometimes people fill the gap with plugins, but it often feels like Cypress core just doesn’t have the resources to keep pace with modern web development.

I submitted an uncontroversial PR to Cypress a year ago that they still haven’t acknowledged. I suspect that they just don’t have the resources to review external pull requests.

In contrast, Playwright has just 603 open bugs despite receiving roughly the same volume of bug reports. When I filed a bug with Playwright, they triaged it and gave me a meaningful response in less than one business day.

Playwright integrates better with VS Code 🔗︎

Playwright offers an official VS Code plugin, which gives you context-aware auto-complete. It’s something I never realized I’d been missing from Cypress until I saw it in Playwright:

Playwright’s VS Code plugin offers context-aware auto-complete.

In Cypress, there are a small number of functions, and you exercise different functionality by passing special string values. It’s hard for IDEs to help with those semantics, but Playwright’s list of explicit TypeScript functions make it easier for the IDE to help you out. There are third-party VS Code plugins for Cypress but nothing the Cypress team officially supports.

Parallel tests are free in Playwright 🔗︎

In theory, you can run parallel tests for free in Cypress, but they deliberately make it inconvenient. I can’t blame them, as parallel tests are one of the flagship features in Cypress’ paid SaaS tool, so they lose money by making the free version more useful.

Microsoft has vastly deeper pockets than Cypress, so they can afford to give away all of Playwright’s features for free. As such, Playwright supports parallel tests out of the box.

What I miss about Cypress 🔗︎

Cypress’ syntax is more consistently fluent 🔗︎

Both Cypress and Playwright offer fluent-style APIs, where you chain together a series of actions into a single statement.

Cypress more strictly adheres to the fluent style, allowing the developer to read testing logic left to right.

cy.get(".navbar-item [data-test-id='log-in']").should("be.visible");

With Cypress, the order I write the code matches the order I think about the test. First, I grab a reference to the element. Then, I think about what assertions I want to make.

In Playwright, the ordering is a little muddled. Before I start locating the element I want to test, I have to wrap the code in an expect call:

await expect(
  page.locator(".navbar-item [data-test-id='log-in']")
).toBeVisible();

Playwright’s syntax interrupts the left-to-right ordering I’m used to from Cypress. I wish Playwright’s syntax looked more like this:

// INVALID - not how Playwright actually behaves
await page
  .locator(".navbar-item [data-test-id='log-in']")
  .expect()
  .toBeVisible();

Cypress has a small, independent team 🔗︎

I have a personal appreciation for Cypress as an open-source company, and in particular, Gleb Bahmutov, their VP of Engineering. Gleb publishes high-quality blog posts, and he’s an excellent conference speaker.

When I wrote a blog post about Cypress, Gleb was gracious in sharing feedback to improve the post. After I published it, Cypress promoted my article on their blog.

Microsoft, on the other hand, has historically has been hostile to open-source. They’re in a period of friendliness now, but if the winds change, and they realize they can make more money by crushing open-source, they probably will.

If this were a movie, Cypress would be the scrappy underdog you can’t help but root for, and Microsoft would be the reformed villain who’s probably going to betray the hero in the third act.

Cypress test artifacts work in CI 🔗︎

When a Cypress test fails, it screenshots your app at the point of failure and saves the image to disk. It’s easy to configure your CI platform to keep these images as test artifacts for easy debugging. Similarly, Cypress lets you save videos of each of your tests that you can also publish as CI test artifacts.

Cypress produces test artifacts that are easy to view as CI artifacts

Playwright produces a more complicated set of test artifacts. Instead of simple images and videos, Playwright generates a static web app for viewing all the test artifacts.

Unfortunately, Playwright’s report viewer doesn’t work on CircleCI, so I have to download assets and run a Playwright server locally instead of just viewing them from my CircleCI dashboard.

Cypress’ Docker image actually contains the software 🔗︎

In a pattern I’ve only ever seen in end-to-end testing tools, the official Docker images for Cypress and Playwright don’t actually contain the tools themselves. That is, the Cypress Docker image does not contain Cypress, and the Playwright Docker image doesn’t contain Playwright.

Instead, the Docker images contain the dependencies you need to install Playwright or Docker respectively. So when you’re running the Playwright Docker image, you still have to install Playwright as part of your environment setup.

There must be some good reason for this, but I’ve never understood it. When I complained about this to the Cypress team, they added a special cypress/included image that contains the Cypress tool itself. There doesn’t seem to be an equivalent Docker image for Playwright.

Summary 🔗︎

Even though I’m only a few hours into using Playwright, I found it to be a substantially better experience than Cypress. Between the clearer APIs, simpler testing setup, and speed, I’m likely 50-100% more productive in Playwright than I was in Cypress.

Going forward, I’ll be testing all of my new apps with Playwright. I’ll likely even port some of my old Cypress tests to Playwright for apps where my tests have crept above the five-minute mark.

If you’re a Cypress user, I strongly suggest giving Playwright a look. To me, the jump from Cypress to Playwright is as substantial as from Selenium to Cypress.