Back to roadmap
Module 8 · Caching, Queues, Async WorkDay 07525 min

Message Brokers

Decouple producers and consumers with durable queues.

← PreviousNext →
Day 075

Message Brokers

Producer
service
Kafka topic
queue
Group A
service
Group B
service
Signal path
Kafka topic with multiple consumer groups
Producer
service
flow
Kafka topic
queue
Kafka topic
queue
flow
Group A
service
Kafka topic
queue
flow
Group B
service
Memory hook

Message Brokers: decouple producers and consumers with durable queues

Mental model

absorb bursts before they become outages

Design lens

Kafka requires real ops; SQS is one click.

Recall anchors
KafkaRabbitMQSQS

Why it matters

Brokers buffer and deliver messages between services. Kafka is a partitioned commit log: high throughput, ordered per partition, replayable. RabbitMQ is a flexible AMQP broker. SQS is a managed simple queue.

Deep dive

Kafka: producers write to partitions; consumer groups read with offsets. Replays are first-class.

RabbitMQ: routing patterns (direct, topic, fanout) and acks. Lower throughput than Kafka.

SQS: best for simple work queues; FIFO variant for ordered, lower throughput.

Demo / scenario

Async order processing.

  1. Order placed → produce 'OrderCreated' to Kafka.
  2. Multiple consumer groups read: payments, inventory, analytics.
  3. Each group has its own offsets — no coupling.
  4. Replays for recovery: rewind offsets.

Tradeoffs

  • Kafka requires real ops; SQS is one click.
  • RabbitMQ has flexible routing but lower scale.
  • Stream vs queue is a fundamental shape choice.

Diagram

Producer
Kafka topic
Group A
Group B
Kafka topic with multiple consumer groups.

Mind map

Check yourself

Loading quiz…

Sources & further reading

← PreviousNext →