199 Tasks Done, 1 Task ForeverSpark · easy · ~10 minA daily join between events and accounts used to finish in 6 minutes. Today it's been running for 45. The Spark UI shows the final stage is 199 of 200 tasks complete — but one task has been running for 40 minutes with no end in sight. Cluster CPU is mostly idle.Figure out why one task refuses to finish.Evidence

Stage Tasks

Spark UI

Job Snippet

Checkpoints

What is happening?

Which fix directly addresses the skewed key?

Why does simply raising shuffle.partitions NOT fix a single hot key? Answer interview-style.