Back to roadmap
Module 8 · Caching, Queues, Async WorkDay 07825 min

Outbox Pattern and Idempotency Keys

Atomic publish-with-state, deduped consumption.

Day 078

Outbox Pattern and Idempotency Keys

App
service
DB + outbox
datastore
Broker
queue
Consumer
service
Idempotency KV
datastore
Signal path
Outbox + idempotency consumer
App
service
flow
DB + outbox
datastore
DB + outbox
datastore
flow
Broker
queue
Broker
queue
flow
Consumer
service
Memory hook

Outbox Pattern and Idempotency Keys: atomic publish-with-state, deduped consumption

Mental model

absorb bursts before they become outages

Design lens

Outbox is operational overhead.

Recall anchors
OutboxCDCIdempotency keys

Why it matters

Outbox saves the message in the same DB transaction as the business state, then a separate process publishes it. Combined with idempotency keys at consumers, it gives a robust pipeline.

Deep dive

Idempotency keys live in a KV with TTL; consumers check before processing.

Change Data Capture (Debezium) reads DB WAL and publishes to Kafka — generalized outbox.

Beware long-running transactions blocking outbox publish.

Demo / scenario

Migration from sync HTTP to events.

  1. Add outbox table + worker.
  2. Move call-site to enqueue event instead of HTTP call.
  3. Consumer service uses idempotency on event_id.
  4. Reliability up; no double-execution.

Tradeoffs

  • Outbox is operational overhead.
  • CDC simplifies producers but couples to DB internals.
  • Idempotency storage adds latency.

Diagram

App
DB + outbox
Broker
Consumer
Idempotency KV
Outbox + idempotency consumer.

Mind map

Check yourself

Loading quiz…

Sources & further reading