Intention

Allow E2E tests built through the Postman tool to be included in a GitHub Actions pipeline.

Thus, the way of building E2E tests becomes broader, not only limited to code, but also to a visual approach.

In the end, there is the versioning of collections and test environments, in addition to their integration in the GitHub Actions pipeline.

Scenery

I sampled the entire content of this publication in the sample repository of my last publication: Hexagonal Architecture Distilled in JavaScript — Ultimate Guide.

This repository contains a backend of a Blog written in JavaScript, running in Node.js. Nothing so sophisticated.

What will we exercise in this publication:

• Versioning of the Postman collection and environment.
• Assertion writing in tests using the Postman API.
• Running versioned tests using Newman, Postman’s headless CLI.
• Integration of tests that run by Newman, in the GitHub Actions pipeline.

Structuring the Collection and Versioning

As stated in the opening topics, I will use the Hexagonal Architecture Distilled in JavaScript — Ultimate Guide publication repository as an example.

I suggest that the reader download the content from the repository and import the collection and environment into Postman. Both are versioned in the directory ./e2e/postman/**.

Preparing the Test

We commonly use tools like Postman, Insomnia, etc., to demonstrate example cases of API routes.

We will do the same thing. However, with the addition of an assertion that will run after Postman’s request ends.

In the “Tests” tab, illustrated in Figure 1.0, we have access to the Node.js runtime to perform our tests.

The testing tool is an API from Postman itself, with the syntax on top of the ChaiJS BDD — official Postman documentation: Writing tests | Postman Learning Center.

In this test example, I add two expectations, serving as an acceptance of what this route needs to provide.

We added a similar strategy in a product at Bom Pra Crédito. Such assertions respect the acceptance contained in the User Stories.

Notice that in line 12, I add a conditional to an assignment. Therefore, I add the Post ID inserted in the base, in environment, so that the next tests that need this fixture can use it.

When running the test script with Newman, it’s important to remember that the tests happen in the order they are in the menu layout.

In this case, I first perform the “Create a Post” test, using the assignment in line 13 to prepare the environment for the next “Get a Specific Post” test.

Versioning the Tests

Following the model I practiced in the examples repository, I export both the test collection and the environment variables. Both are located in ./e2e/postman/**.

By performing the versioning of these two files, we were able to provide future engineers working on the team with examples of how to use the project routes with ease.

Another point is that having these files saved in the project’s central repository will be essential for the strategy of running the tests with Newman and GitHub Actions.

We could carry out the testing strategy in the pipeline, sharing these files by link.

But that way we wouldn’t be able to leave the shared information in our version control system. Therefore, I proceed with the initial form proposed in this publication.

Test Script and Pipeline Integration

Now that we have the structured and versioned tests, we move on to the last part of the publication.

Let’s declare the script that will run the test script with Newman and finally integrate it into the GitHub Actions pipeline.

Introducing the Newman

It is a CLI that runs the collections defined by Postman headless.

Thanks to this tool, we will run the script for independent testing of graphical dependencies.

First of all, beef up this amazing project with a ⭐️ in your GitHub repository.

To consult the documentation made available on the Postman website, go to: Running collections on the command line with Newman.

Installing and Defining the Script

First, let’s install Newman as a development dependency in our project, using the npm i -D newman command.

After installation, we need to declare a script in package.json that runs Newman considering the versioned collection and environment.

Looking at line 19 of the file ./package.json in the example repository, we have the desired script.

"test:postman:headless": "newman run ./e2e/postman/hexagonal-architecture-distilled.postman_collection.json -e ./e2e/postman/hexagonal-architecture-distilled-dev.postman_environment.json"

In the first input of the script, we have the location of the test collection. In the second, the environment we will run.

As mentioned before, we could inform the links to the respective files in the cloud. It’s up to you.

Structuring the GitHub Actions Pipeline

Until this part of the publication, we already have automated tests with Postman being versioned and ready to run with Newman in our repository.

Now, let’s upload our application infrastructure into a container within the GitHub Actions flow (currently running on Microsoft Azure infrastructure).

In the example application, we use MongoDB for data persistence. We would need a temporary container that has MongoDB to run the E2E tests.

To perform the definition of containers within GitHub Actions, we use Service Containers.

Using the services definition, we can specify which Docker images to use in this workflow.

Following the example of the file .github/workflows/acceptance-tests.yml in the example repository, we can see that in line 13, we defined a container with the official image of MongoDB.

Looking at the official GitHub Actions documentation, if our Service Container is specified within the workflow itself, we need to perform the explicit port mapping.

On line 18, I map that host port 27017 will be random within the Docker container.

Thus, the host understands the communication with MongoDB through port 27017, serving as if it were an alias.

The result of this strategy is in line 29, where I assign the environment variable MONGO_URL to mongodb://mongo:${{ job.services.mongodb.ports[27017] }}/hexagonal-blog.

The result of the variable contained within the interpolation, accessed through the “alias” 27017, at runtime will be the port of the Docker container.

Changing ports is the result of container volatility. As its lifetime is execution oriented, we have a different communication port for the MongoDB service every time the workflow runs.

Finally, in the step of line 26 “Run Server and Test with Newman”, I use the command npm run start to raise the server at the address specified in env.HOST_URL and env.PORT, and concatenate it with the script npm run test:postman:headless to perform the tests with Newman.

Golden Tip: Important to remember that the cold start of this repository of examples is practically null. In a serverless application with several plugins for example, we would need to chain after npm run start, a command that provides a delay.
Example: npm run start & sleep 30 npm run test:postman:headless.

Conclusion

We implement everything that was proposed at the beginning of the publication: Tests created by Postman running in GitHub Actions pipeline independent of graphical interface.

An empowering reading for the publication’s theme is: Continuous Delivery Pipelines: How To Build Better Software Faster, Dave Farley.

In a Continuous Delivery pipeline, these E2E tests would be in the Acceptance stage.

We could chain other types of automated tests that provide the necessary security for the project to always be in the release stage.

But that’s a subject for another publication.

References

• Postman
• Newman is a command-line collection runner for Postman (github.com)
• Expect / Should — Chai (chaijs.com)
• Features • GitHub Actions
• Mongo — Official Image | Docker Hub
• Insomnia
• Continuous Delivery Pipelines: How To Build Better Software Faster, Dave Farley

More content at plainenglish.io