Kafka in ProductionHow Uber, PayPal, Agoda, and 30+ companies actually run Kafka — scale, failure, cost, and the judgment calls.Week 2: Consumers under fire: lag, rebalances, failoverThe canonical outage chain — slow sink → GC pause → missed heartbeat → rebalance storm — and why failback, not failover, is the real DR interview question.

Figure 1 — the canonical cascade. By the end of this week you can read this as a prediction: which link breaks first in your own consumers, and which config governs each arrow

Friday, 4 p.m., a routine deploy. Twenty minutes later the lag graph goes vertical. Kubernetes says every consumer pod is healthy. The application logs are clean — no errors, no exceptions, nothing. But consumption is zero, and it stays zero while lag climbs into the millions. Nothing crashed. So why is nothing moving?This week is the consumer group under fire: the failure chain that runs through *your* code, *your* JVM, and *your* Kubernetes settings before it ever touches a broker — and the disaster-recovery question (failback, not failover) that most designs skip.After this week you canThe full week 2 brief is part of LeetData Pro.Case studies

PayPal — Scaling Kafka Consumer for Billions of Events

Sophie Blee-Goldman — Kafka Streams and Rebalancing through the Ages

Agoda — How We Solve Load Balancing Challenges in Apache Kafka

Uber — Disaster Recovery for Multi-Region Kafka at Uber

Agoda — How Agoda Handles Kafka Consumer Failover Across Data Centers

Etsy — Adding Zonal Resiliency to Etsy’s Kafka Cluster

Exercises

Lag: Slow Consumer or Producer Spike?

Rebalance Storm — Scale Out or Tune?

Broker Down — Live Recovery (Simulator)

Leader Lost at Peak — Availability vs Data Loss

The Failback

Week 2 check: Consumers Under Fire