Airflow at scale, backfills and idempotency, DAG discipline, and the in-house engines — Netflix Maestro to Pinterest Spinner.
Orchestration is where data engineering's software-engineering discipline shows: Shopify's multi-tenant Airflow lessons, Snap's multi-cluster DAG switching, Netflix rebuilding Maestro's engine 100× faster, and the backfill/idempotency machinery every senior interview probes.
Reliability at four altitudes: the scheduler's own hot path, staging-then-promote gates, product health above task status, and config-driven fleets — because execution success is not data correctness.
The dashboard is green. Every DAG succeeded last night; the on-call slept; the platform's uptime metric reads 99.8%. And this morning the VP of Sales is in your team's channel because the revenue dashboard is *wrong* — quietly wrong, three days running. The pipeline that feeds it ran perfectly: on time, exit code zero, all checks green… all *two* of them. Somewhere between "every task succeeded" and "the numbers are right," a gap opened wide enough to drive three days of bad decisions through. Whose job was it to close it?
The full week 3 brief is part of LeetData Pro.
