Distributed transactions

The moment a local transaction stops being enough

Suppose an operation updates two databases. A local database transaction cannot make both updates atomic because each database controls its own commit. A distributed transaction protocol or an application workflow has to coordinate the outcome.

Wrong "If one request fails, call rollback on the other database."

Right A failure can happen after one side commits but before the caller receives its response. The caller cannot infer the remote state from a timeout.

How two-phase commit reaches one decision

Two-phase commit (2PC) uses a coordinator and a set of participants.

1. During prepare, the coordinator asks every participant whether it can commit. A yes vote means the participant has recorded enough state to commit or roll back later and promises to follow the final decision.
2. If every participant votes yes, the coordinator decides commit. If a participant votes no or the coordinator cannot collect every yes vote, it decides rollback.

Preparation is more than a health check. Oracle Database records redo information and keeps distributed locks when a node prepares. The prepared node then waits for the global decision.

The awkward failure is a coordinator crash after a participant has prepared. The participant cannot safely choose an outcome from silence. A commit decision may already exist or may have reached another participant. Until recovery resolves the in-doubt transaction, locks can remain held and conflicting work can block.

Wrong "A timeout makes rollback safe."

Right A timeout detects missing communication. It does not reveal whether the transaction committed elsewhere.

This cost grows with the work inside the transaction. Oracle's microservices guidance warns that prepared locks remain until commit or rollback and can become a performance bottleneck. That is the practical reason teams avoid stretching 2PC across independently operated services, especially when calls can be slow or participants do not share one transaction technology.

Sagas commit locally and compensate later

A saga splits one business operation into local transactions. Each service commits its own change and triggers the next step through a message or event. If a later step fails, compensating transactions address the work that already completed.

For an order flow, the forward and recovery paths might be expressed as:

reserve inventory -> authorize payment -> request shipment
release inventory <- void authorization <- shipment rejected

Compensation is business logic. Canceling a reservation may release stock, while compensating a charge may issue a refund. Microsoft documents an important limit: compensation does not necessarily restore the exact state from before the operation because concurrent work may have changed the same data.

Compensation can fail too. A recoverable workflow records progress and reuses the same operation identifier when it retries a compensation. The participant should treat an already completed identifier as success:

async function compensate(command: CompensationCommand) {
  await database.transaction(async (tx) => {
    const prior = await tx.compensationLog.find(command.operationId);
    if (prior?.status === "completed") return;

    await applyBusinessCompensation(tx, command);
    await tx.compensationLog.markCompleted(command.operationId);
  });
}

A saga also lacks built-in isolation across services. Other work can observe intermediate local commits, and concurrent sagas can produce lost updates or nonrepeatable reads unless the design adds suitable controls.

Choreography and orchestration

With choreography, each service publishes events and reacts to events from other services. There is no central controller. Microsoft describes it as a good fit for simple workflows with few services, but the event dependencies become difficult to track as steps are added and can form cycles.

With orchestration, a controller directs participants, records workflow state, and handles recovery. This makes a complex flow easier to follow, at the cost of coordination logic and a component whose failure must be handled.

The distinction is about where workflow control lives. Both styles still need correct local transactions, reliable progress tracking, and compensations where forward progress cannot continue.

TCC makes the reservation explicit

Try-Confirm/Cancel (TCC) fits resources that can be held in reserve. In Oracle's protocol, Try creates reservations, Confirm consumes every accepted reservation, and Cancel releases them. Oracle uses a seat with AVAILABLE, RESERVED, and SOLD states as the concrete model.

Try:     AVAILABLE -> RESERVED
Confirm: RESERVED  -> SOLD
Cancel:  RESERVED  -> AVAILABLE

TCC differs from a general saga because the participant exposes a provisional state before the final outcome. The application must define what the reservation means. A payment authorization and an inventory hold fit this shape; an arbitrary external effect does not become reservable merely because the caller names a cancel endpoint.

The reservation itself has a cost. The resource is unavailable to competing work until confirm or cancel releases it. A TCC implementation may initiate cancellation after a configured timeout, but each participant still has to implement cancellation so that it releases the reservation. TCC is useful when that bounded hold is acceptable and participants can implement the reservation lifecycle honestly.

Blocking is a knowledge problem

After a participant votes yes in two-phase commit, it has entered a prepared state. It can carry out either final decision, but it cannot choose between them safely. If the coordinator disappears, the participant lacks one fact: was commit durably decided?

This is why retry and timeout do not remove the core blocking case. They may restore communication or trigger recovery, but silence alone is compatible with different histories. The coordinator may have failed before recording commit, after recording it, or after sending it to only some participants.

Wrong "The coordinator is a single point of failure, so elect another one and abort."

Right A replacement coordinator must recover an authoritative decision or run a termination protocol that preserves the decision already possible from participant states. Election by itself supplies no transaction outcome.

While the answer is unknown, prepared resources remain costly. Oracle documents that prepared nodes retain locks and that an in-doubt transaction can require manual resolution when the underlying failure cannot be repaired quickly. PostgreSQL gives the same operational warning for prepared transactions: they keep their locks and should be closed out promptly by an external transaction manager.

What three-phase commit adds

Skeen's canonical three-phase commit (3PC) inserts a committable "prepare to commit" state between the uncertain waiting state and commit. Reaching that state implies that every participant voted yes. Under the paper's failure assumptions, a backup coordinator can collect operational participants' states, move them to a common state, and complete the protocol consistently.

The claim needs its original assumptions attached. Skeen assumes point-to-point communication remains possible between operational sites, the network never fails, and site failures can be detected and reported reliably. Under that model, the added phase produces non-blocking protocols for the covered site failures. The paper also notes that the extra phase adds a substantial message overhead.

Wrong "3PC fixes 2PC during any network partition."

Right Its non-blocking result depends on communication and failure-detection assumptions that a partitioned asynchronous deployment does not satisfy.

This makes 3PC valuable for understanding the blocking boundary, but its theorem is not a license to decide from arbitrary timeouts. Fault-tolerant commit protocols based on consensus address a different model. Gray and Lamport describe classic 2PC as the zero-fault-tolerant form of Paxos Commit, while Paxos Commit uses consensus to tolerate coordinator failures. More replicas and messages do not remove the need to state the failure model.

Why 2PC is avoided across services

2PC provides a clean atomic outcome when every resource supports the protocol and the transaction manager is operated as part of the system. Its cost is coordination on the synchronous path plus a prepared interval in which participants retain durable state and locks. A failure can extend that interval far beyond normal request latency.

Oracle's microservices documentation recommends sagas to reduce the locking period to each local transaction and cites 2PC's extended locking period and performance constraints. This is a narrower and more defensible statement than "2PC does not scale." Scale depends on transaction duration, contention, participant count, failure rate, and the transaction manager. The protocol becomes unattractive when independent services would share a wide failure domain or when some stores cannot participate in the same atomic commit protocol.

Sagas exchange that coordination cost for application work. They expose intermediate commits, require compensations, and need monitoring of a workflow that may outlive one request. Microsoft lists dirty reads, lost updates, and nonrepeatable reads as possible saga anomalies because there is no built-in isolation across participant data.

Designing the recovery path

A compensation should produce a valid business state. It may not restore the byte-for-byte starting state, and it may need to account for concurrent changes. Because compensation can itself fail, its progress has to survive a worker restart and retries have to recognize completed operation identifiers:

async function runCompensation(step: CompensationStep) {
  const state = await workflowStore.read(step.sagaId, step.id);
  if (state === "completed") return;

  await participant.compensate(step.operationId);
  await workflowStore.markCompleted(step.sagaId, step.id);
}

async function compensate(operationId: string) {
  await database.transaction(async (tx) => {
    if (await tx.completedOperations.has(operationId)) return;
    await applyBusinessCompensation(tx);
    await tx.completedOperations.add(operationId);
  });
}

For a workflow with an irreversible step, define a pivot. Before the pivot, completed work is compensable. After it, remaining work should be retryable so the operation can reach a consistent final state. Microsoft describes retryable post-pivot transactions as idempotent.

An orchestrator is often clearer once recovery depends on explicit ordering, stored workflow state, or a pivot. Choreography avoids a central controller, but each service must understand the events that advance or compensate the flow. Microsoft calls out cyclic dependencies and difficult workflow tracking as choreography grows.

TCC is narrower than a saga

TCC asks each participant to create a real reservation during Try. After all reservations are accepted, the coordinator confirms them all or cancels them all. Oracle's MicroTx documentation allows an initiator to leave reservations for a configured timeout to cancel. The application still has to implement cancellation so that it releases the resources, and it defines what each reservation means.

This can shorten the uncertain business state by moving validation into Try. Oracle's seat example also obtains payment authorization during Try so Confirm is less likely to fail. The price is capacity held in RESERVED state, plus application logic for expiry, confirm, and cancel.

Wrong "TCC is 2PC implemented with HTTP verbs."

Right Both coordinate a final outcome, but TCC exposes application-level reservations and application-defined confirm or cancel operations. 2PC prepares enlisted transactional resource managers and keeps their transaction state until the atomic decision.

Choose TCC only when every participating effect has an honest provisional form. A seat, inventory quantity, or funds authorization can be reserved. An irreversible notification or completed physical shipment cannot be made provisional by protocol naming.