Contract testing

Why a shared integration environment stops scaling

The usual way to catch cross-service breakage is a pre-prod environment that runs every service together, so any team can deploy and see if it broke a neighbor. This works at five services and falls apart at thirty. The environment is shared mutable state: one team's half-finished deploy fails everyone else's tests, so a red board rarely means your change is bad, and releases queue behind whoever broke it last. You also need every service booted, seeded, and healthy at once just to test one request path between two of them.

Contract testing removes the shared environment from the boundary check. Instead of running the consumer and provider together, each side tests alone against a recorded agreement about the requests and responses that pass between them.

What contract testing is

The dominant tool is Pact, and its model is consumer-driven. The consumer writes ordinary unit tests, but the HTTP client under test talks to a Pact mock provider instead of the real service. In each test the consumer registers an expected request and a minimal expected response, sends a real request to the mock, and asserts its own code handled the response. Passing tests emit a pact file: a JSON document listing each interaction the consumer exercised.

{
  "consumer": { "name": "checkout-api" },
  "provider": { "name": "ledger-service" },
  "interactions": [{
    "description": "a charge against an existing account",
    "providerState": "account 42 exists with a positive balance",
    "request":  { "method": "POST", "path": "/charges",
                  "body": { "accountId": 42, "cents": 500 } },
    "response": { "status": 201, "body": { "id": "ch_1", "status": "posted" } }
  }]
}

That file is the contract. It is published to a broker, a shared store both teams read. The provider then runs verification in its own pipeline: the broker hands it the pact, and it replays each recorded request against the real running provider, checking that the response contains at least the fields the consumer said it needs. Consumer and provider never run together, yet you learn whether they agree at the boundary.

One word on "consumer-driven": the contract is the union of what consumers use, not the provider's whole API. Martin Fowler's phrasing is that consumer expectations collectively form the provider's contract. So the provider learns exactly which parts of its interface are depended on, and which it can change freely.

Provider states: setting up data before the replay

A recorded request like POST /charges for account 42 only verifies if account 42 exists in the provider when the request replays. That precondition is a provider state. The consumer names it in the interaction (given "account 42 exists with a positive balance"), and the provider registers a setup hook under that exact name that seeds the data before the interaction runs.

The point of naming states is isolation: each interaction is verified on its own with no data carried over from the previous one, so verification does not become a fragile ordered script.

Wrong "A provider state is the consumer telling the provider what request to send." A provider state is about the provider's data, not the request. It answers "what has to be true in the provider for this interaction to make sense," such as which account exists, and the provider owns the setup code. Right the consumer supplies a name; the provider maps that name to real setup (insert a row, stub a downstream call, force a 500). Nothing about the request or the consumer's own state belongs in it.

What a green contract does and does not prove

Contract testing sits between unit and end-to-end. It verifies the boundary between two services, replacing the need to run them together, but it is not an integration run of the whole system and it is not functional testing of the provider.

Wrong "Our contract test is green, so checkout and the ledger work together, and we can drop the end-to-end suite." A green pact proves the shapes line up for the interactions the consumer exercises. It says nothing about whether a real purchase debits the right amount, whether a three-service journey completes, or whether the provider's business logic is correct. Right keep functional tests on the provider and a thin end-to-end test over the real journey. Contract tests let you delete the shared environment used for boundary confidence, not the coverage that checks the system does its job.

can-i-deploy and the broker matrix

The part teams get wrong first: green pipelines are not the deploy gate. A consumer build passing its Pact tests and a provider build passing verification tell you those two builds are internally fine. They do not tell you the version you are about to ship is compatible with the version currently running in production.

The broker records every result as a cell in a matrix: rows are consumer versions, columns are provider versions, cells say whether that pair verified. You also tell the broker what is deployed where, so it knows which versions to check against.

# after a successful deploy, record it
pact-broker record-deployment --pacticipant checkout-api \
  --version $GIT_SHA --environment production

# before the next deploy, ask the matrix
pact-broker can-i-deploy --pacticipant checkout-api \
  --version $GIT_SHA --to-environment production
# "Computer says yes \o/"  -> exit 0, deploy proceeds
# "Computer says no"       -> exit 1, deploy blocked

can-i-deploy checks the candidate version against the versions deployed in the target environment. If checkout-api adds an interaction the deployed ledger-service has not verified, the gate fails even though both pipelines were green. This is what lets two teams release on independent cadences: neither can ship ahead of a boundary the other side has not caught up to.

Wrong "can-i-deploy checks that the latest consumer and latest provider both passed." It checks against what is deployed, not what is newest. The newest provider may verify your contract while the version in production does not, and it is production you are about to break.

Consumer-driven scope: what breaks verification and what does not

A contract covers only the interactions the consumer exercised, so compatibility is asymmetric. The provider can add an optional response field or a whole new endpoint and no contract notices, because no consumer asserts those are absent. Remove or rename a field a consumer reads, and that consumer's interaction fails verification.

That asymmetry is the feature. The provider gets fine-grained insight into which parts of its interface carry real usage and which are dead, so it can evolve the unused parts freely and knows exactly who to talk to before touching a used one.

Wrong "Any change to the provider's response breaks all the consumer contracts." Additive changes are safe by construction; only removals and renames of used fields break, and only for the consumers that use them. A team that believes every change is breaking will version the API on every release and lose the main benefit.

Provider verification runs against real code

Verification replays the recorded requests against the running provider, not against a spec or a mock. This is the difference that makes Pact more than schema-matching: it catches a provider whose implementation has drifted from what it claims to return.

The cost is coupling through provider states. Every named state the consumer declares needs matching setup in the provider, so a consumer can add an interaction that fails the provider build until the provider team writes the state handler. States that seed shared data or hit real downstreams are a common source of flaky verification. Keep setup hooks writing directly to the data store, and keep each state self-contained, since interactions are verified in isolation with no carryover.

Verification usually runs on a webhook: publishing a changed contract fires the provider's build so it verifies against the new expectations without manual coordination. Contracts and results are versioned by git sha, which is what makes the matrix meaningful across branches and environments.

Consumer-driven vs bi-directional, and OpenAPI diff as the lighter alternative

Consumer-driven Pact needs the provider team to run verification. Sometimes you cannot get that: a third-party API, a team that will not adopt Pact, a provider you only consume. Bi-directional contract testing (a PactFlow feature) is the fallback. Each side produces an artifact independently: the consumer emits a pact from its mock as usual, and the provider supplies an OpenAPI spec that its own team verifies against its real API with whatever tool it already uses (Dredd, ReadyAPI, Postman). At can-i-deploy time the platform statically compares the two, checking the consumer's pact is a valid subset of the provider's spec.

The tradeoff is real and worth stating in an interview. Bi-directional never executes the consumer's requests against the provider, so its guarantee is weaker than replaying them. It trades runtime verification for a lower barrier to entry and reuse of existing tools.

At the lightest end, if you only want to catch breaking API changes, diff two OpenAPI specs in CI:

oasdiff breaking ledger-v1.yaml ledger-v2.yaml
# exits non-zero on breaking changes: removed endpoint, removed
# response property, new required request field, changed type

This is not contract testing (it knows nothing about which interactions a consumer uses, so it flags changes no consumer depends on and cannot confirm the provider matches its own spec), but it is cheap backward-compatibility insurance where full Pact is not justified. The event-side analogue, schema-registry compatibility for Kafka messages, lives in the kafka-internals module.

Where Pact does not fit

Pact's own docs list the boundaries, and naming them is a senior signal. It fits when you control both sides and both are under active development, with good communication between the teams. It does not fit for a public API whose consumers cannot be individually identified, for a provider team that will not run verification, for pass-through APIs that just forward to a downstream, or as a general browser mocking tool. And it verifies contract compliance; it says nothing about correctness or performance: a provider can honor every contract and still compute the wrong balance or fall over under load.