2. About me
Principal Software Engineer at Dell-EMC
Organizer Cork Software Crafters meetup
Paulo Clavijo @pclavijo - January 2018
Cork Software CraftersPaulo Clavijo Esteban @pclavijo
paucls.wordpress.com
github.com/paucls
3. Context
● < 2014 - Monolithic systems
○ Unit tests, Integrations Tests and slow Webdriver E2E tests.
● 2014 - Transition to Microservices architecture and web apps
○ Services tested with Unit tests, Component tests.
○ Web apps with Unit tests and E2E tests using a stub backend.
○ System Integration tests limited to high-value User Journeys.
● 2016 - CD Pipelines
○ Teams doing Continuous Delivery. Feature Toggles.
○ Continuous Deployment only to Staging.
● 2017 - Implemented Consumer-Driven Contract Tests
○ Solved the REST APIs integration issues. Full CD to Production.
Paulo Clavijo @pclavijo - January 2018
4. Context
A previous attempt of using Contract Tests didn’t work out
● Contract testing purpose and value was not understood properly.
● Introduced partially without bringing real value.
○ Only consumer contracts for Java services without corresponding providers test.
○ Not used on Web apps.
○ No integration with CI/CD pipeline, pull requests.
○ Overhead of writing them didn’t pay off.
● Tests scope was too broad.
● Shared client libraries. No Front-end first / CDC flow.
Paulo Clavijo @pclavijo - January 2018
5. The Challenge
Multiple feature teams, applications, microservices. Independent
deployments, ...
… REST APIs integration issues.
How to test collaboration between system components?
● Isolated Component Tests, Test Doubles and the Test
Symmetry problem
● Integration Tests?
● System End-to-end tests?
Web-app
to Service
Service to
Service
Paulo Clavijo @pclavijo - January 2018
6. Contract Tests
Traditionally understood as a separate set of tests that checks that all the calls against
your test doubles return the same results as a call to the remote service would.
Consumer
Remote
Service
Test Double
Unit tests Contract tests
Paulo Clavijo @pclavijo - January 2018
7. Consumer-Driven Contract Tests
Unit tests in both sides where consumers detail clearly what are the interactions and
behaviours expected from the provider API.
Consumer Provider
Consumer
Contracts
Test Double
Verify on providerRecord consumer
expectations
Unit tests Unit tests
Paulo Clavijo @pclavijo - January 2018
8. Consumer-Driven Contract Tests
Consumer Driven Contracts (CDC) is a pattern that drives the development of the Provider
from its Consumer's point of view. It is TDD for services.
It is possible to make changes on a provider knowing that consumers won't be impacted!
Consumer A
Consumer B
Consumer C
Provider
Consumer-Driven
Contract
{ Id: 100,
name: “A task” }
{ Id: 100,
name: “A task”
expedite: true }
{ Id: 100,
duration: 20 }
{ Id: 100,
name: “A task”,
expedite: true,
duration: 20 }
Consumer Contract A
Consumer Contract B
Consumer Contract C
Paulo Clavijo @pclavijo - January 2018
9. Consumer-Driven Contract Tests
They do not test the behaviour of the service in depth but that the inputs and outputs of
service calls contain required attributes.
https://martinfowler.com/articles/microservice-testing
Paulo Clavijo @pclavijo - January 2018
10. Consumer-Driven Contracts
Great for developing and testing intra-organisation microservices.
● You (or your team/organisation/partner organisation) control the development of both the
consumer and the provider.
● The requirements of the consumer(s) are going to be used to drive the features of the
provider.
● There is a small enough number of consumers for a given provider that the provider team can
manage an individual relationship with each consumer team.
https://docs.pact.io/documentation/what_is_pact_good_for.html
Paulo Clavijo @pclavijo - January 2018
11. Pact
● Some of the tools that support Consumer-Driven Contract Testing are Pact and
Spring Cloud Contract.
● We chose Pact because:
○ Generation of contracts from the code
○ Provider verification (pact-jvm-provider-junit)
○ Supports multiple languages JVM, Node, JS web apps, …
○ Pact Specification
○ Pact Broker
Paulo Clavijo @pclavijo - January 2018
12. Pact Broker
● Helps managing, versioning and distributing
contracts.
● Great tool to visualize contracts and system
relationships (dependency graph).
● Contracts as a form of “Live Documentation” for
APIs and interactions between components.
Paulo Clavijo @pclavijo - January 2018
14. CDC on the Consumer side
Consumer
Contracts
Provider
Contract
Tests
1) Define expected interactions
2) Tests the API Client
code on the consumer
Provider
Contract
Tests
Consumer
Contract
Tests
4) Generates contracts
Consumer
API Client
Pact Mock
Service
3) Verify interactions
Paulo Clavijo @pclavijo - January 2018
15. CDC on the Consumer side
1) Define expected interactions
2) Tests the API Client code
3) Verify interactions
Paulo Clavijo @pclavijo - January 2018
16. CDC on the Consumer side
4) Pact Mock Service generates contract
files for successful tests
These contracts, in a CDC flow, can
now be shared and used to drive the
development of the provider.
Paulo Clavijo @pclavijo - January 2018
17. Contract Testing on the Provider side
Consumer
Contracts
Provider
A
P
I
Provider
Contract
Tests
1) Pact auto-generates provider tests
3) Tests the provider
Provider
Contract
Tests
Provider
Contract
Tests
2) Set-up provider state
Paulo Clavijo @pclavijo - January 2018
18. Contract Testing on the Provider side
Auto-generated Pact provider contract tests
Paulo Clavijo @pclavijo - January 2018
19. Paulo Clavijo @pclavijo - January 2018
Contract Testing on the Provider side
Pact uses the state description on the
contract to determine which set-up logic
needs to be executed before running a
generated test.
24. Defining Interactions
When we define the expected interactions between a consumer and a provider, the ‘state’ and
‘uponReceiving’ descriptions used should be written in a consistent style.
Later on, on the provider side, Pact uses the uponReceiving description to name the auto-generated
test, and the state description is used to find a corresponding test state.
// Expected Interaction
provider.addInteraction({
state: 'a task with id task-id exists',
uponReceiving: 'a request to delete task task-id',
withRequest: {method: 'DELETE', path: '/tasks/task-id'},
willRespondWith: {status: 204}
});
Paulo Clavijo @pclavijo - January 2018
25. Defining Interactions
Examples
Successful interaction where data is returned
state: ‘tasks for project with id project-id exist’,
uponReceiving: ‘a request to get tasks for that project’
Successful interaction where no data is returned
state: ‘tasks for project with id project-id do not exist’,
uponReceiving: ‘a request to get tasks for that project’
404 error interaction
state: ‘project project-id does not exist’,
uponReceiving: ‘a request to get tasks for that project’
Domain validation error interaction
state: ‘a task task-id exists and domain rules exist that prevent its
modification’,
uponReceiving: ‘a request to update that task’
Successful interaction where data is created
state: ‘a project with name project-1 does not exist’,
uponReceiving: ‘a request to create a project with name project-1’
Paulo Clavijo @pclavijo - January 2018
26. Live Documentation
Contracts are not only used by the tests, we also intent them to be user-friendly. Developers
can consult contracts in the Pact Broker who shows them in a nice documentation style.
Paulo Clavijo @pclavijo - January 2018
27. CDC in our pipeline
Paulo Clavijo @pclavijo - January 2018
28. CDC vs Contract Tests
TDD is more than having Unit Tests …
… CDC is more than having Contract Tests
Think in it as Outside-In TDD for a whole system!
Is what we do, a technique and attitude, discipline on the
design and development flow.
Paulo Clavijo @pclavijo - January 2018
29. What is next?
● Pact v3
○ Contract testing for messages (for services that communicate via event streams and message
queues)
● Pact v4
○ Matching times and dates in a cross-platform manner
○ Additional Matchers
● … Beyond REST APIs. GraphQL?
○ Servers publish a statically typed system specific to their application, and GraphQL provides a
unified language to query data within the constraints of that type system.
Paulo Clavijo @pclavijo - January 2018
30. Further reading
● Implementing Consumer-Driven Contract tests in Angular, Paulo Clavijo
https://paucls.wordpress.com/2017/08/04/pact-consumer-driven-contract-imple
mentation-in-angular
● Configure Pact JS Consumer Contract tests in Angular, Paulo Clavijo
https://paucls.wordpress.com/2017/08/04/configure-pact-js-consumer-contract-
tests-in-angular
● Sample project on Provider Side
https://github.com/paucls/task_list_api-kotlin-pact
Paulo Clavijo @pclavijo - January 2018
31. References
● ContractTest, Martin Fowler. https://martinfowler.com/bliki/ContractTest.html
● Pact documentation https://docs.pact.io/documentation/implementation_guides.html
● Testing Strategies in a Microservice Architecture, Toby Clemson
https://martinfowler.com/articles/microservice-testing/
● Integrated Tests Are A Scam, J.B. Rainsberger
http://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam
● Consumer-Driven Contracts: A Service Evolution Pattern, Ian Robinson
https://www.martinfowler.com/articles/consumerDrivenContracts.html
● Microservices, Flexible Software Architecture. Eberhard Wolff.
● Contract testing with GraphQL, Andreas Marek
https://www.andimarek.com/post/contract-testing-graphql
● https://github.com/DiUS/pact-jvm/tree/master/pact-jvm-provider-junit
Paulo Clavijo @pclavijo - January 2018