Green test pyramids with Cypress – UI testing of the future

Cypress is a young open-source testing framework for web-based user interfaces (UI). Cypress tests are written in JavaScript and, as is also common with Selenium-based technologies, are based on the Document Object Model (DOM) of the HTML of a web application.

Compared to its well-known ancestor, Selenium, Cypress offers many improvements that lead to increased stability, reliability, run-time, and development speed. The advantages of the youngster and why it is worth using Cypress will be discussed in more detail in the following article.

Sharp pyramids: UI over end-to-end tests

Before writing the first Cypress tests with motivation, it should be decided what will be tested at all. You should consider whether only the UI or the whole system end-to-end (E2E) should be tested. The classical test pyramid considers both kinds of testing as coexisting. It is common to use few but business-critical E2E tests and many detailed user interface tests, decoupled from any backend services. Also, when looking at the cypress tree, the framework’s namesake, this approach is confirmed. Tapered test pyramids indicate many unit, UI and integration tests and few E2E tests.

What’s wrong with executing all of our test cases E2E against our entire systems? Development teams often have to experience this in a painful way. E2E tests are slower, as requests are made to “real” backends by the UI under test. They are also less stable, as tests depend on data storage of the backends. In real environments, data is dynamic – a difficult situation for automated testing. Data changes through synchronisation from external systems, manual user interaction and is also made available to consumers with eventual consistency. In order to react to this in our test code, waiting and repetition mechanisms are often built in, which increases the complexity and frustration in the team. As a result, E2E tests rarely provide deterministic results, but rather fail more often, which can hinder the development process. E2E testing is required for critical functionality but comes with a big price tag.

Pure UI tests, in which the backend requests are mocked by controllable data, are better suited to test each functionality in detail. They can be developed test-driven in combination with the UI and are a cheaper investment in a well-tested code base.

Controlling network traffic with Cypress

In pure UI tests, you benefit from control over data that is normally provided by the server side. Cypress shines here in every respect, as its architecture (visible in the next image) takes this into account to its advantage.

Cypress’ architecture during a test consists of a Node.js process (the “server side”) and a running browser. The browser, instrumentalised by Cypress, shows a page that embeds the tested application as well as Cypress as an iFrame. Test code is also executed in the Cypress iFrame, so that the test code and application code run in the same browser tab and thus in the same JavaScript loop. The test code communicates with the Node.js process via WebSockets. The Node.js process acts as a proxy for every HTTP request of the browser and can even execute shell commands .

The key to controlling network traffic is the proxy. HTTP requests are intercepted, which makes it possible for Cypress tests to wait for specific requests to be fulfilled. HTTP responses can also be completely overwritten with fixtures instead of sending the requests to the real target and waiting for their real response. This allows us to fake responses without having to configure the application against another API host – this is all done based on the proxy, if we tell Cypress to do so with the right commands .

Besides HTTP requests, local time and even JavaScript objects can be overwritten during runtime (stubs) or just observed for verification purpose (spies). This is useful when testing time- or date-dependent parts of our application. By overwriting JavaScript objects of the application, we can even make it think that a user is logged in. This lets us simplify and speed-up testing in a way that we can concentrate on testing our application’s core functionality.

No more Selenium WebDriver for modern web a,pps

Independent of the type of testing, Cypress trumps with high stability and reliability compared to Selenium-based tests. Selenium users know the randomly appearing and frustrating “flakiness” of DOM access and assertions which lead to failing tests. Cypress is not based on Selenium WebDriver as a way to access DOM. It was built completely from scratch, based on principles that are better suited for the concepts of todays web applications. JavaScript-heavy sites and Single Page Applications (SPA) continuously fetch resources via HTTP requests or WebSockets, trigger DOM changes and thus are always in motion.

It is very hard to test these dynamic applications using concepts like simple “waits” and “sleeps”. Instead, Cypress found ways to wait on command executions to be finished, elements to be present and done with animations. HTTP requests can be waited on to make sure that the application has finished reacting on user input. These improvements make it easier to write clean, readable and less complicated testing code. Tests’ execution time can be reduced to the minimum time that is required for the application to transit all statuses under test. It is no longer slowed down by arbitrarily long “sleeps” and “waits”.

DOM changes are also recorded by Cypress as snapshots. This means that even after the test has been executed in graphical, interactive mode, it is possible to see how exactly the application has changed over the course of the test. Additionally, videos of entire tests as well as screenshots of failures can be recorded. This is very useful when investigating errors during test execution in continuous integration pipelines. Even visual regression tests are possible and well-supported with Cypress . With this kind of testing, one can verify that an application’s visual representation is exactly reproduced compared to its specification based on a snapshot.

Flora and fauna – extendability of Cypress

Cypresses shape the landscape of the beautiful Tuscany, but they are by no means the only thing. The flourishing ecosystem around Cypress continues to develop. This is encouraged by the framework itself, which can be extended by custom commands, plugins, and events by the developers.

There are many publicly available Cypress plugins for you to make use of. They cover login for E2E tests, ways of doing component testing, file uploads, drag-and-drop, and many more. Another good example of Cypress’ extendability is its integration with Cucumber in order to write BDD-style tests using Gherkin syntax.

Cypress custom commands are also great to use within your own code base. It’s a good way of extracting code that we need more than once. The following code example shows how we can mock the response of the Cognito identity provider by AWS to simulate a successful login request.

The ecosystem also includes the Cypress.io dashboard service . This service offers various features regarding recording, inspecting and analysing your tests’ results. It is the only paid component of Cypress.io but also offers a free tier for small projects with limited builds per month. Unfortunately one needs to use the dashboard service when it comes to running tests in parallel. This limitation applies because Cypress learns about the execution time and behaviour of your test specifications and persists them on the server side. It collects this data to ensure an even parallelisation over multiple workers. Interestingly there is an alternative project called “Sorry Cypress ” which follows the mission of providing an unpaid alternative to the official dashboard service. “Sorry Cypress” is open source and can be operated on your own. Meanwhile this project is also financed via a paid cloud service that you can use.

Conclusion

Cypress undoubtedly solves many annoying problems of web-based interface testing and lets developers get to the heart of the matter. In many digital product development teams, UI and E2E testing is not yet given sufficient priority. Automated testing of UIs, while theoretically seen as promising, is often perceived as complex, time-consuming, and not robust enough to invest into. Cypress is on the right way to level up the reputation of UI testing. This leads to a growth of awareness and enables many teams to benefit greatly.

Teams that need to support a variety of (mostly outdated) browsers may view Cypress’ restricted browser support as an issue. As of writing this in October 2020, all Chromium-based browsers (incl. Microsoft Edge and Electron) as well as Mozilla Firefox (in beta support) are supported. Internet Explorer and Microsoft Edge 10 (legacy, non-Chromium-based) fall off the wayside.

For teams that need to support old browsers like Internet Explorer, Cypress’ browser support can be a problem. As of now (July 2021) Chrome, Chromium, Firefox, Edge and Electron are supported by Cypress – in this respect, Cypress has evolved in a positive way.

It is also often criticised that running tests in parallel is only possible when using the Cypress.io dashboard service. However, “Sorry Cypress” may provide an alternative if you’re willing to operate such a service yourself. “Sorry Cypress” is now also available as a cloud service for a monthly fee, which is kinda funny looking at their initial mission.

In conclusion, Cypress is fully recommended for UI and end-to-end testing if you can do without legacy browsers.