From zero to start writing end-to-end tests

When it comes to testing big applications, we need to test several modules or parts associated with one another.

Usually, these modules are tested independently (e.g., unit tested), but that does not necessarily mean that they work as expected when combined.

For testing these kinds of situations, we may use integration testing or end-to-end (E2E) testing.

Integration testing often entails evaluating a certain function that is dependent on other functions. Integration testing aims at testing the connection and communication of different software modules or components.

As for end-to-end testing, you are looking to test the application the same way a real user experiences it. You want to make sure everything (dependencies, environment, database, UI, and so on) works as expected. End-to-end testing aims at testing functionality and performance. Again, this is all done in an automated manner.

This guide aims to help understand how to set up and run end-to-end tests using Cypress against the Meteor framework. You’ll find the corresponding code in our example repository.

Outline

What is end-to-end testing?Why use Cypress?Installing Node.js and MeteorInstalling CypressFiles created by Cypress on the first runConfiguration and setup of Cypress in a Meteor appRunning testsCustom Cypress commands for querying elementsCustom Cypress commands for MeteorUsing fixturesTest coverageConclusionNext steps?

What is end-to-end testing?

Software may be tested at various levels. The unit test is the most basic level. In unit testing, we write code to test the correctness of a function or routine (a “unit”). Unit tests should be fast. When run, they give us immediate feedback on whether a routine is working as expected (for example, returning the expected output for defined input).

However, confirming that all of those minor bits operate correctly in isolation might be insufficient. It does not guarantee that they will operate properly together. Our app still might break. Integration or end-to-end tests come into play here.

Usually, integration tests are performed before end-to-end tests. Having a set of integration tests might help reduce the number of end-to-end tests. This is recommended since integration testing is faster to write “theoretically” and more efficient.

End-to-end tests simulate a user’s step-by-step experience. They allow you to validate different parts and layers of the application. They help gain confidence about the intended functionality of the application’s crucial capabilities, such as connecting with other systems, interfaces, databases, networks, and other applications.

Each type of testing (unit, integration, end-to-end) takes a different approach but they all have the same goal: to determine whether the outcome of some code execution meets a certain expectation. They are complementary in their purpose, and when combined, they can provide you with confidence that your application is running as intended.

Why use Cypress?

Cypress is a Javascript end-to-end testing framework that allows you to easily write tests for user interfaces. It facilitates tests that simulate normal user interaction in a web application. Cypress provides a plethora of tools like commands, plugins, time travel, screenshots, videos, etc., that will help you write robust tests faster.

Selenium was the go-to tool for end-to-end testing as it has been around for nearly 17 years. During that time, the web — and with it, the requirements for testing — have changed dramatically. Cypress is a testing framework designed specifically for the asynchronous nature of today’s web. It operates within your application and provides full control over it.

Furthermore, what I like best about Cypress.io is the huge community backing it, the abundance of recipes, well-written documentation, and its public roadmap.

Limitations of Cypress

Be that as it may, there are some limitations to Cypress, too. Be aware of them before you jump onto a new tool for your stack.

You cannot use Cypress to drive two browsers at the same time.It doesn’t provide support for multi-tabs.Cypress only supports JavaScript for creating test cases.Cypress doesn’t provide support for browsers like Safari and IE at the moment.Limited support for iframes (but it’s on the roadmap).

Installing Node.js and Meteor

Meteor is a free and open-source JavaScript web framework written on top of Node.js with many cool features such as reactivity, hot code push, a client-side database, and client/server code reuse capabilities.

In this article, I assume that you are already familiar with Meteor, have a working Meteor development environment, and a working Meteor app.

For the sake of explanation, we’re using the React todo app from Meteor. But in general, you can use Meteor with any frontend framework. You can still follow this guide, even when you don’t know about React.

Node.js and npm (or yarn) should also be installed on your machine and available from the command line. If you haven’t already, get it from the official website or use a handy tool like n.

Installing Cypress

First, let’s install Cypress. I’ll be integrating it into a todo app built with Meteor and React from Meteor’s tutorial series. If you’d like to try out Cypress in a clean app, you can download the todo app and start from there (you’ll need to run npm install and might want to run meteor update -all-packages to get the newest release). But you can also follow this guide to integrate Cypress into your own Meteor app.

cd /your/project/path
npm install cypress –save-dev

It will take a few minutes to install the packages as well as the Cypress binaries required to run tests. The folder structure we need for Cypress will be created when we run it for the first time (more on that below).

Next, let’s add npm scripts to packages.json to run Cypress, which we’ll use throughout this article.

https://medium.com/media/e8df00202bb236af9b9b77140c883524/href

After adding these scripts, you can run npm run cypress:ui (or yarn cypress:ui) for a graphical test runner and npm run cypress:headless (or yarn cypress:headless) to run it in headless mode (i.e., on the console or in a CI environment).

Run npm run cypress:ui to open the Cypress UI and spend some time getting to know the test runner.

Next, we’ll have a look at some new files and directories that Cypress produces after running one of these commands.

Files created by Cypress on the first run

You’ll discover a new directory cypress/ and a new cypress.json config file at the root of our project.

cypress/
├── fixtures/
├── integration/
├── plugins/
└── support/
cypress.json

cypress/fixtures

In the fixtures folder, you may save static data that will be utilized throughout our tests. We won’t spend much time discussing this area of Cypress, but if you want to learn more, I recommend looking at the official documentation about fixture files.

cypress/integration

This folder is where all of our tests will live. The sample tests that you experimented with earlier will all have been generated here.

cypress/plugins

Cypress is running a node process that can be used to extend some of its functionality. This is done via plugins. It includes configuring file pre-processors or loading configurations, both of which will be covered below.

cypress/support

Out of the box, Cypress provides the cypress/support/index.js file that will be run before every test. It can be used for several things such as a global beforeEach hook overrides and setting up custom commands which I will also demonstrate as part of this article.

Configuration and setup of Cypress in a Meteor app

Let’s delete all example tests generated by Cypress and write our first test.

# Delete all Cypress example tests
rm -rf cypress/integration/examples

Configure test directories in cypress.json

Meteor by default loads every .js file in our project directory (except for specific folders likeimports/). Attempting the same with our new tests files will result in an error:

While processing files with static-html (for target web.browser):
…
Your program is crashing. Error while waiting for a file change.

To avoid this, do one of the following:

Move the cypress directory to tests/cypress/ (a tests folder will be ignored by Meteor by default), oradd cypress/* to .meteorignore (to tell Meteor to ignore this folder)

You will have to edit cypress.json accordingly. In the remainder of this article, I assume that you went with option (1) and your Cypress folder is now located under tests/cypress in your project.

https://medium.com/media/6fbc1ab64097b89ad53354683ec81ae9/href

You can find more details about these configurations in the official Cypress docs.

Create a new folder tests/cypress/integration/tests so we can add some extra logic (e.g helper functions …) in tests/cypress/integration without Cypress detecting it as actual tests.

Adding a placeholder for the first test

Now we are ready to create our first spec file. Create a login.spec.js file in tests/cypress/integration/tests to test the login feature.

https://medium.com/media/19f350c86497d25527b28fb705102adb/href

Now run npm run cypress:ui, select our newly created spec file login.spec.js and you should see this result:

Way to go!

Have a dedicated environment for running tests

When running tests, you should do so on a separate database. I find it handy to use Meteor’s local database that is created when starting a local server. Even when you are developing your app on a local database, you can create another database as a dedicated test database.

To do so, Meteor has an undocumented (only appears in the changelog) environment variable METEOR_LOCAL_DIR which specifies the directory from which Meteor launches everything. By default, it is set to .meteor/local. When changed to .meteor/test, the test instance will be completely isolated from the development instance, including the database.

Also, I prefer to run Meteor on a different port than in development, so we have a completely separate environment to work on. Let’s add the script for this in package.json:

https://medium.com/media/7a8d5519f9ee421124b7d47cc82e2c36/href

Besides METEOR_LOCAL_DIR, we set two more environment variables:

BABEL_ENV: This will be used to load the “test” configuration in .babelrc which we’ll need for test coverage.DISABLE_REACT_FAST_REFRESH: This will disable hot module replacement, which we won’t need for running tests. You will only have to set this when using React and the react-fast-refresh package.

Run and stop Meteor automagically

You still have to manually launch Meteor (npm run e2e) and run your tests in a different terminal (for example, npm run cypress:headless). To make this more convenient, you can install start-server-and-test (npm install –save-dev start-server-and-test). As the name suggests, this package starts the server, waits until it’s ready, and launches the tests in one command. So in our package.json we now should have:

https://medium.com/media/afd6708f9da732babe419d2cce9af599/href

Run npm run test:dev:headlessand the server should start and Cypress will run after that.

Fetching DOM elements in Cypress

While using Cypress, one best practice for querying DOM elements is to add data-* attributes to these elements, rather than using class or tag selectors.

Anti-Pattern: Using highly brittle selectors that are subject to change.Best Practice: Use data-* attributes to provide context to your selectors and isolate them from CSS or JS changes.

To avoid repeating ourselves, we create a helper function testProp and some constants that hold the IDs that we’ll add to the elements to test.

Create a new file imports/testIds.js:

https://medium.com/media/97c4788e0ebe10b2f67554f3570511e0/href

We’ll do the same for our app’s routes. It’s likely that your test scenarios require visiting routes throughout your app. Let’s add a new file imports/routes.js:

https://medium.com/media/056b8b3003ff20bff9bb477ee6d373ed/href

Let’s import everything from testIds.js and routes.js into a new file tests/cypress/integration/constants/index.js. This is to make sure that we have all our test dependencies in one place.

https://medium.com/media/ed0116c66430721fe43630361a342235/href

Whenever we need to access an ID or a route in a test, we import it from tests/cypress/integration/constants.

In the LoginForm component (of the example todo app), add the ID to the element using our helper function:

https://medium.com/media/60ef3bf7a4f738aa57c8fac71d9f33bc/href

By using testProp (instead of directly setting data-test-id) we have the flexibility to easily change the data attributes in the future and to only set these attributes when the app is running in testing mode (and hide them in production).

Seeding the database

Usually, when testing, it’s preferred to have a clean state of the database for each test. The database should have a defined set of data to test against.

By resetting the database before each test, we avoid:

having inconsistent data throughout our test suites,tests which depend on previous tests (which can be hard to reason about),complicated test scenarios and dependencies,unstable and unreliable test data.

There are several ways to fill the database with initial data:

Create a test that interacts with your UI so that data is created that then will be used by the following tests.Expose Meteor methods to set up a certain test on the server. This results in “backdoor” functionalities that you don’t want to expose in production.We can simply use fixtures for filling out forms, setting user credentials, and most of the data that a user might put into your application.Use a Cypress task. In a task, you can execute code in Cypress’s Node.js process. We can leverage this API to tear down or seed a database with data we’re in control of.

Using a Cypress task is arguably the cleanest approach and is also recommended by the Cypress documentation. It provides flexibility, a good developer experience, and isolates the seeding from the rest of the code. However, resetting the database before each test this way is certainly not the fastest approach and it might slow down your test suits.

There is a helpful library called mongo-seeding that manages to seed the database for us (assuming you’re using Meteor with MongoDB). All we have to do is provide the JSON files for the documents that we’d like to put into the database and implement the seeding as a Cypress task.

npm install –save-dev mongo-seeding

First, let’s create a tests/cypress/plugins/data directory that contains a tasks and a users directory (same as the collection names “tasks” and “users” in our todo app example). Then add a tests/cypress/plugins/data/tasks/tasks.json and tests/cypress/plugins/data/tasks/users.json file:

https://medium.com/media/127c86564e443037ed1c9f645706e183/href

Next, let’s create a seeder.jsfile in/tests/cypress/plugins :

https://medium.com/media/fdf87d8012fa4c5d768646d856ca6da5/href

Then, import this seeder as a Cypress task like so:

tests/cypress/plugins/index.js

https://medium.com/media/3adb6023e5face7ebcfb3cbcd48986f6/href

Now, whenever we need to seed our database we just call the task in the beforeEach hook:

https://medium.com/media/a5883edae6619cb45998e78087545285/href

Running tests

Let’s improve the login.spec.jstest to see Cypress in action.

https://medium.com/media/fb26302649b9fccd93dc1c1bb8b44c74/href

Now, run npm run test:dev:headless (or yarn test:dev:headless) in your console, and the test should pass successfully.

Custom Cypress commands for querying elements

cy.get or cy.visit are examples of built-in commands in Cypress.

The ability to create custom commands is another feature of Cypress that improves the developer experience and test reusability. This is especially handy for routines such as logging in.

The supportdirectory is a good location to put our custom commands.

For this example, I am going to create a custom command cy.getById(ID_STRING) to query an element by its data-test-id attribute.

in tests/cypress/support/commands.js

https://medium.com/media/cc6a247053df3ce581777a3c201c9ed0/href

We can now rewrite our test to look like this:

https://medium.com/media/fdbab5e1c8e8e7e0c8b8872d623e913c/href

Now that you get the gist of what commands are, let’s see how we can write some that will help us test in Meteor.

Custom Cypress commands for Meteor

cy.getMeteor()

As you noticed in our login.spec.js test we run cy.window().its(‘Meteor’).invoke(‘user’) to get the window object, get Meteor from it, and then call the Meteor.user() function to assert that the user is authenticated.

As you will use the Meteor object often, this will be tedious. You probably guessed right — we can put this in a custom command.

in tests/cypress/support/commands.js

https://medium.com/media/2c1f0ce0d150bd4d1fcea9e9b4b6cdcd/href

Now we can update the tests/cypress/integration/tests/login.spec.js:

https://medium.com/media/aaedfc618d7201d8d8661990fa2ab522/href

cy.allSubscriptionsReady()

Another support command that we need is waiting for Meteor subscriptions to be ready after a page load.

in tests/cypress/support/commands.js

https://medium.com/media/496be4f1095d43d70f82a284b27e5d90/href

cy.visitAndWaitForSubscriptions(…)

We need this mostly when visiting a URL within our app. So let’s create another custom command that waits for all the subscriptions after visiting a URL.

in tests/cypress/support/commands.js

https://medium.com/media/eac0d0702f2a24e8029ad64288ccbd10/href

Now, we can be sure that after visiting a route we have all subscriptions ready before we make any assertions.

cy.callMethod(…)

One other important command that we’ll add is cy.callMethod. It calls a Meteor method by passing it the method name and parameters :

in tests/cypress/support/commands.js

https://medium.com/media/8cf2d96f0cc1ad5580daa2144f181968/href

Finally login.spec.js will look like this :

https://medium.com/media/18f4e20fbc3e96eb09d6197f505a7f44/href

Using Fixtures

Fixtures are a way to load predefined data during a test. They are complementary to seeding the database before each test as they are used within a test.

Accessing aliases as properties with this.* will not work if you use arrow functions for your tests or hooks.

To be consistent in our test suits, we use the regular function () {} syntax as opposed to the lambda “fat arrow” syntax () => {}. Sometimes we’ll need to use this to access a Cypress feature in a test which is only possible with the function syntax.

First, let’s create our fixtures file:

tests/cypress/fixtures/tasks.json

https://medium.com/media/b8cbe0482b73e5540eb6be7ea77d69a6/href

Let’s create a new test spec file for tasks called tests/cypress/integration/tests/tasks.spec.js where we will use fixtures and our custom command cy.callMethod().

https://medium.com/media/73b54f225b35b5f48631ae413ba130ab/href

As you noticed in the beforeEachhook we fetch the fixtures and assign an alias for it named tasks. To get the first task in fixtures we use this.tasks.testUserTask[0].

Test coverage

In terms of quality and efficacy, test coverage (how many lines of code your tests cover) is an essential metric in software testing.

This metric is computed using the following formula:

We need to utilize specific tools to identify the lines of code that were executed when running our test cases. Essentially, each line of code is provided with a counter of how often it was run. This is called code instrumentation.

Luckily, Cypress provides a library @cypress/code-coverage to automate this. In the background, it uses istanbul.js for instrumentation and generating code coverage reports.

For more about test coverage, the official Cypress documentation about test coverage is as usual pretty good.

Test coverage on the client

Now, let’s add test coverage to our app. The first step is to install the @cypress/code-coverage library:

npm install –save-dev @cypress/code-coverage

To set it up, add totests/cypress/support/index.js

https://medium.com/media/c170699b2ccaad7fab20de7f11e35fd4/href

Adapt your tests/cypress/plugins/index.js

https://medium.com/media/3adb6023e5face7ebcfb3cbcd48986f6/href

Next, we must add the following dependencies to our project:

npm install –save-dev @babel/preset-env babel-plugin-istanbul
npm install –save-dev @babel/preset-react # Only for React, ymmv

Since Meteor transpiles new JavaScript features out-of-the-box, we usually don’t need to care about setting up Babel. However, here we make sure that the instrumentor istanbul.js also understands the source code of our app.

These dependencies are used to configure the preprocessor to transpile our source code to the instrumentor.

Additionally, we configure Babel in a .babelrc file at the root of the project.

Remember BABEL_ENV? By setting BABEL_ENV=test, we ensure that the following presets and plugins are only used in a test environment.

https://medium.com/media/3c156619b1efd37525cdfcc78c19bd84/href

istanbul.js provides a command-line tool nyc that is used for instrumenting the code. To configure that to our needs as well, add nyc.config.js to the root of your project.

https://medium.com/media/7110900d6dbf46a7fff16bf17c41f2d5/href

Now run the test.

A .nyc_outputdirectory will be created in our project root folder containing the raw coverage data from nyc.A tests/cypress/coveragedirectory containing the reports is created:

You’ll see a graphical representation of the code coverage results when you open tests/cypress/coverage/index.html.

Test coverage on the server

One final but important step is to also collect server code coverage. Right now, Cypress cannot know what happens on the server until we provide an interface to retrieve this data. To do this we hook into Meteor’s WebApp and expose a route to fetch the __coverage__ object. Note that we only expose the route when the app is running in a test environment.

In server/coverage.js add this code:

https://medium.com/media/fc757591903e5d7ff90ab87f24d29c27/href

Then import this file to server/main.js (or wherever your server entry point lives).

https://medium.com/media/66558550e5a3c5d26f548dde8b470e5e/href

Now add the config necessary for Cypress to detect and merge it with the client coverage.

In cypress.json add :

https://medium.com/media/8b0bee6e051228c619eb6cd0b6bb5a50/href

After running the tests again, the report should like this with new files detected from the server:

Conclusion

It’s time to rejoice! By now you should’ve set up Cypress in your Meteor app, be able to generate coverage reports, understand how custom commands make your life easier, and be ready to test away.

You can view all that was said in this article in this example repository. If you have suggestions on how to improve this guide, please let us know.

Next steps?

Now that you’ve set up everything, you can start writing meaningful end-to-end tests for your app. Code coverage reports can give you pointers where tests are missing. Apart from that, the official Cypress documentation will often come to the rescue.

Resources

This guide wouldn’t have been possible without the documentation, posts, discussions, and ideas of the open-source community. Here are the original sources.

How to write end-to-end test for Meteor with CypressCode Coverage with MeteorJS and CypressOfficial Cypress documentationMeteor commands threadhttps://github.com/coleturner/solidselectorsCypress hooksCode Coverage Issue

This article was written by Idriss Mahjoubi and Timo Schneider from consider.ly. If you like it, tell your friends about it 😊

Testing a Meteor app with Cypress was originally published in Meteor Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.