Test automation in microservices architecture with integration and end-to-end tests
Since the first industries the humans have been trying to improve products quality. Using quality assurance paradigm, having individual or sampling tests, we have been trying to ensure the quality meets the customer expectations.
In software engineering is not different. Will this deploy break something? Will the customer face some side effect issue? Those are common questions when we speak about releasing new code to production in software development.
The test methodologies and tools have been popularised over the years and today many tech companies follow the “You build it, you run it!” philosophy, in which every Software Engineer take ownership of certain parts of the system, since coding till the deploy phase, and so take the responsibility of ensure the code meets the customer needs, often using automated tests to validate it.
Automating their tests allows teams to know whether their software is broken in a matter of seconds and minutes instead of days and weeks. (Martin Fowler)
When it comes to the practical field, there are many tools and many paradigms to handle tests in software engineering: we have unit tests, integration tests, interface tests, end-to-end tests and so on, it may even depend on the technology you’re using.
One famous picture can be highlighted in this context though: the Mike Cohn’s test pyramid. It introduces the idea of test categories and how we could distribute our set of tests for each category.
It might not be 100% always true and it may change according the context and the business of each team and company, but it does give us an overview about the options we have and how could we distribute the tests in our systems.
In the base of the pyramid we have unit tests, the ones that are cheap to write and to run, they test isolated pieces of your system. In the middle of the pyramid there are the service tests, also known as integration tests, they can be employed to test on application level if some endpoint of your service behaves as expected for example. In the very top we have the UI (or end-to-end) tests, they are used to test entire sections of the system or entire features, they often involve more than one service.
Unit tests are very common nowadays and they are quite well known by developers. The challenges are bigger when we get on service (or integration) tests and UI tests (or end-to-end ). This last two will get special attention in this article: how to use them and some paradigms and frameworks we have been using in Klar.
Unit tests
Uni tests according Mike Cohn are the base of the pyramid. They can be used to test every small piece of your code, every business logic for each class or each function.
The positive point about them is that usually they run very fast so you have a quick feedback either they passed or not. In another hand they give you a very fragmented view of the code quality, not saying anything about the overall system when it gets integrated. Mocks for external calls are welcome here.
We have a vast number of tools for unit tests. In Java world there are frameworks like JUnit, Hamcrest. For JS we can mention Jest. For React Enzyme and many many others.
Integration tests
Integration tests are in the middle of the Mike Cohn’s test pyramid. We want to test some integration between pieces of the system, not the whole system yet though. Good examples of integration tests are tests for controllers and entrypoints of your application. You can simulate someone interacting with a specific API and asses the if it behaves as it should.
Let’s take a look in some code examples here.
For Java, in special within Spring world, a very good set of tools is provided to help writing integration tests.
However often the applications have external dependencies, such database. A nice library for Java to help in this case is the Test Containers. It provides a way to integrate normal docker containers in you application’s context while running integration tests. So you can have a real Postgres database or a Kafka instance running for example while your integration tests run.
Since they are docker containers, the accuracy of the tests is higher, as you can run your tests in a very similar environment as if the application would be running in production, rather than if you use mock or internal solutions.
But how Testcontainers looks like in a real application?
First you need to add the Testcontainers dependencies, let’s take an example an application that needs a Postgres database and Kafka.
Once you have the dependencies added, you can create a configuration class to be imported in the tests, where you specify the container setup. You can also do things like override environment variables in order to make your application point to the new and fresh containers.
Once you have it setup, you can add this configuration class in your integration tests class:
With this small setup you ensure that when the test is ran, first the Testcontainers will download and spins the containers for your test.
You can see a complete example of a integration test using it here: BalanceControllerIT.java
You can also do things like test Kafka consumption of messages. You can assert for instance that when an event is sent and consumed by your application, it does process it accordingly your expectation. The setup is the same and you can find out a complete example here: TransactionConsumerIT.java
As many tech choices, the Testcontainers has its drawbacks. If in one hand you have a more real setup in your tests, from another side the tests may take longer to run. Every time you want to run the switch of tests, the containers need to be spin off, which may take some seconds. But you also have some good alternatives if you want to go with faster tests rather then accuracy as embedded-database and embedded Kafka from spring-kafka-test.
Other two useful frameworks that may help you to build integration tests are:
- Localstack: mock AWS environment.
- MockServer: mock API calls to external services.
End-to-end tests
End-to-end tests are the highest layer in the Mike Cohn’s pyramid. They are kind of expensive to run in a sense they will need all components of a system to be up and running in order that they can be performed. But they are very powerful if you take in consideration that you can test your entire system or big features. They often help to validate the interaction of services and how a set of microservices for example behave in a particular scenario. They can also help in report API change breaks between services.
Specially in a time when the services are every day smaller and the features are more distributed over multiple services, the e2e tests take an important role to ensure a complete feature is working before a deploy.
In Klar we have been using for end-to-end tests in the backend a framework called Karate. It’s an open source project from Intuit that allows you to interact with Rest APIs and offer an easy sintaxe to make assertions. It follows the idea (and the syntax) from BDD and Cucumber.
In Karate you define a Background section in which you can setup some constrains. Then you have the Scenarios, which are the interactions that you want to add inputs and assert the results.
Let’s take a look in another code example. Assume we have a financial application with two services: one is responsible for transactions and another takes care of user’s balance. Every time a new transaction is created in the transaction service, balance service is notified through a Kafka message and the user’s balance should be updated.
Taking in consideration that they are 2 separate applications, how could we test this feature? Here is an example using Karate:
Here we make an interaction with the transaction service, publishing a new transaction, and later in the second scenario we assert some expected behaviour in the balance service, if the balance has been updated.
The test code can be found in transactions.feature.
End-to-end tests are really good when we want to test high level integration between parts of the system.
Conclusion
We walked through different test paradigmas in this text, in special integration tests and end-to-end tests. How we can use them and when, since each type of test has some costs and drawbacks associated to it. Integration tests are good to test service level behaviours, while end-to-end are more valuable to test interactions between different services, or even different microservices.
Tests can be connected to your deploy pipeline and they offer you a nice layer when you’re introducing new features in your application that things continue to work as they should.
Happy to hear your thoughts about the tools and paradigms I described here and which tools are you using currently to test your software :)
