Partition Pruning — Solution & ExplanationProblems / Partition Pruning / EditorialApproachThe events table contains ~2,900 rows spanning 2 years, but we only need 1 month:

Filter early — restrict to March 2024 before any aggregation. This eliminates ~96% of rows.

Aggregate — GROUP BY event_type with COUNT, COUNT DISTINCT, and AVG.

Order — sort by total events descending.

Why This MattersIn production, tables are often partitioned by date. When the query engine sees a date filter, it can skip entire partitions (files/blocks) without reading them — this is called partition pruning. Even without physical partitioning, a date filter early in the plan reduces the data flowing through subsequent operations.Complexity

With pruning: O(k) where k = rows in March 2024 (~120).

Without pruning: O(n) where n = all rows (~2,900), plus wasted I/O reading 23 months of irrelevant data.

Hints

Filter on event_date BETWEEN '2024-03-01' AND '2024-03-31' before grouping.

Use COUNT(DISTINCT user_id) for unique users.

In PySpark, use .filter() before .groupBy() to prune early.

Solve Partition Pruning yourself →