Scalable Integration Testing for Microservices Deployments
Known for his open source and JavaScript security initiatives, Liran Tal is an award-winning software developer, security researcher, and open source champion in the JavaScript community. He's an internationally recognized GitHub Star, acknowledged for his open source advocacy, and has received the OpenJS Foundation's Pathfinder for Security for his work on Node.js security. His contributions to developer security education include leading OWASP projects, building supply chain security tools, participation in CNCF and OpenSSF initiatives, and authoring books such as O'Reilly's Serverless Security. He leads the developer advocacy team at Snyk.io and is on a mission to empower developers with better application security skills.
Sep 19, 2017 ~ 3 min read
Scalable Integration Testing for Microservices Deployments
share this story on
Many jumped the gun on microservices, and they are ubiquitous today more than ever for implementing service oriented architectures. Moreover, tools and ecosystems like Docker and Kubernetes for container orchestration are driving high adoption for microservices due to good support for orchestration, fast and consistent deployments among other reasons.
If your deployment environment is quite small, lean and CI runs fast in an integrated test environment then chances are that you don’t feel the pain… yet.
Microservices — The More The Merrier?
When your deployment is dependent on many microservices, some of which might turnout to be complicated to deploy, then you feel the pain. Setting up the integrated environment so you can run tests and verify your service works well with others is a challenging task and potentially hinder CI with slowness.
docker-compose and similar tools help in making the job easier, but it doesn’t necessarily scale well for CI when you have tens or hundreds of services, or when services depend on large databases with large migrations backlog, all of which need to be spun up and down during your CI. Your microservice dependencies may have their own service or third-party dependencies, and they evolve and translate into more setup, and changes in deployment which again contributes to the overall problem of maintaining a fully deployed environment for CI.
Consumer Testing To The Rescue
Instead of deploying with all of your service dependencies you would instead rely on verifying an API contract from both the consumer end (you), and the provider end (the service you depend upon)
With the Consumer-Driven Contracts pattern employed, both parties of an API, referred to as consumer and provider, are setting an agreed-upon API contract which they both need to adhere in their testing.
Probably mocking services isn’t news for you, so practically for the consumer this means to set the API contract which will essentially be used as a mocked API service during CI for consumer tests, there-fore eliminating the need for bringing up many service dependencies.
The provider is using the API contract output that the consumer has already created, and executes tests in its own CI, assuring that it didn’t break any API contract for consumers.
this pattern only works well if your teams are collaborating together and aren’t silo’ed. yes, people need to talk, it’s human nature.
Some of the benefits with Consumer-Driven Contracts:
- Setting the API as contract, in advance, even if your provider might not exist or may not have an API developed yet.
- Expectations are defined and set up-front.
- Easy to test, fast to run, lean infrastructure setup.
- Improve collaboration with teams
- When the provider breaks your contract they know about it instantly because they verify their API contract in their CI.
The picture depicts the relationship between the consumer and provider, as illustrated by Andy Kelk in his blog post.
In the picture, a reference to an open source implementation referred to as Pact which we will discuss in a follow-up post on how to do integration testing with the Pact.js project for Node.js microservices.
The Follow-up Post:
