Simpson’s Paradox: When Aggregated Data Tells the Wrong Story

In data analysis, it is tempting to trust the “overall” trend because it looks clean and decisive. Yet some of the most expensive analytical mistakes happen when we summarise data too early and ignore how it behaves inside meaningful subgroups. Simpson’s Paradox is a classic example of this risk. It is a statistical phenomenon where a trend appears in multiple groups of data but reverses or disappears when the groups are combined. Understanding it is not just academic; it directly affects business dashboards, A/B tests, healthcare analytics, and hiring metrics. It is also a topic that often comes up in practical learning environments such as a data science course in Kolkata, where learners are trained to question surface-level averages.

What Simpson’s Paradox Really Means

Simpson’s Paradox occurs when the relationship between two variables changes direction after combining data from different groups. In simpler words: each subgroup says “A is better than B,” but the overall total says “B is better than A” (or no difference at all).

This is not a “math trick.” It is a warning that aggregated results can hide a third variable that shapes outcomes. The third variable is usually a confounder—something that differs across groups and influences the result. When the confounder is unevenly distributed, it can distort the combined average.

A Clear Intuition: Weighted Averages Can Mislead

At the heart of Simpson’s Paradox is the idea of weighting. When you combine groups, you do not simply average subgroup rates; you create a weighted average based on group sizes. If one option appears more often in “harder” conditions and the other appears more often in “easier” conditions, the combined results may flip even if each option performs better within comparable conditions.

Think of two sales teams selling in two regions:

• Region 1 is easy to sell in; Region 2 is difficult.
• Team A performs slightly better than Team B in each region.
• But Team A sells mostly in Region 2 (the hard region), while Team B sells mostly in Region 1 (the easy region).
• The combined close rate may show Team B as better, simply because of where each team operates most.

A Concrete Example You Can Visualise

Imagine two treatments, A and B, used for two categories of patients: “mild” and “severe.” Within each category, A has a higher success rate than B. However, if treatment A is used more often on severe cases (where success rates are naturally lower), and treatment B is used more often on mild cases (where success rates are naturally higher), the overall combined success rate can show B as “better.”

Nothing magical happened to the treatment effectiveness. The patient mix changed the story. This is why analysts must compare like with like before trusting an overall number. In applied training contexts like a data science course in Kolkata, this example is often used to show why stratification and controlled comparisons matter.

Why It Happens: Confounding Variables and Uneven Group Mix

Simpson’s Paradox typically appears due to two conditions:

1. A confounding variable influences the outcome
2. This third variable (severity, region, device type, customer segment, traffic source) affects results and is related to the choice being compared.
3. The confounder is distributed unevenly across the compared groups
4. One group gets more “easy” cases; the other gets more “hard” cases. Even if performance is better inside each subgroup, the overall weighted outcome can reverse.

This is common in real work because processes are not random. Customers self-select plans, marketing channels bring different intent levels, and operational rules route different cases to different teams. Aggregation can silently mix apples and oranges.

How to Detect Simpson’s Paradox in Practice

If you want to avoid being misled, use a consistent checklist:

• Always segment before concluding.
• Break metrics by meaningful dimensions: region, device, customer tier, cohort, severity, channel, or time period.
• Compare rates within comparable groups.
• If you are evaluating two variants, check performance within each segment rather than only the overall metric.
• Look for imbalanced group sizes.
• A large difference in subgroup volumes is a common trigger. The “overall” number may be dominated by one subgroup.
• Use causal thinking, not just correlation.
• Ask: what variable could influence both the choice and the outcome?
• Apply appropriate models when needed.
• Regression with controls, stratified analysis, propensity score methods, or causal graphs can help separate the true effect from confounding.

These habits are a core part of analytical maturity and are often emphasised in a data science course in Kolkata, especially when learners move from simple summaries to real business decisioning.

Conclusion

Simpson’s Paradox reminds us that aggregated data can confidently point in the wrong direction. A trend can appear within every subgroup and still vanish or reverse when those groups are combined, usually because a confounding factor changes the weighting of outcomes. The practical takeaway is simple: do not trust a single overall metric without checking subgroup behaviour, group sizes, and potential confounders. When you build the discipline to segment, control, and interpret results carefully—as you would in a data science course in Kolkata—you reduce the risk of false conclusions and improve the quality of decisions driven by data.