Pipeline · medium · ~12 min
A new DAG shipped at 16:00 with start_date=2024-01-01 and Airflow's default catchup=True. The scheduler did what it was told: it queued ~920 historical runs. Within an hour the warehouse pool is saturated, today's SLA-critical pipelines are stuck behind a wall of 2024 backfill tasks, and the warehouse itself is queuing.
start_date=2024-01-01
catchup=True
Stop the flood without losing the history you actually do want.