Unit 02.02: Aggregation that hides the thing you needed

Aggregation is the most common way to hide the thing you were asked about.

A flat total over two opposite trends

Two regions moving in opposite directions at the same rate.

The code shows the total and then the breakdown.

import pandas as pd

rows = []
for region in ("North", "South"):
    for month in range(1, 7):
        base = 100 if region == "North" else 100
        trend = month * 8 if region == "North" else -month * 8
        rows.append({"region": region, "month": month, "revenue": base + trend})
df = pd.DataFrame(rows)

overall = df.groupby("month")["revenue"].sum()
print("total revenue by month (both regions):")
print(overall.to_string())

print("\nby region:")
print(df.pivot_table(index="month", columns="region", values="revenue").to_string())

print("\nThe total is flat. One region is growing and the other is shrinking")
print("at the same rate, and the aggregate hides both.")

The monthly total is flat across the whole period. One region is growing steadily and the other is shrinking just as steadily, and the sum reports neither.

Nothing here is a data error, and no amount of chart polish would reveal it. The aggregation level is a decision made before the chart, and it determines what can be seen at all.

The mistake this prevents

The mistake is aggregating to the level the data arrives at. Ask what the decision is about - regions, in this case - and aggregate to that, then check whether the total tells the same story as the parts.

Takeaway

Check the total against the breakdown before charting either. Opposite movements cancel, and a flat line is the one shape that can hide arbitrarily large changes.

Mark complete