What Is Simpson’s Paradox in Data Science?

Data can sometimes tell a very different story depending on how you analyze it.

A dataset may show that one group performs better than another when all observations are combined. But after splitting the data into meaningful subgroups, the opposite relationship may appear.

This surprising situation is known as Simpson’s Paradox.

Simpson’s Paradox is particularly important in data science because analysts often work with aggregated data. If important variables are ignored, aggregation can hide relationships that exist within individual groups and lead to misleading conclusions.

In this guide, you’ll learn what Simpson’s Paradox is, how it works, why aggregation causes it, and how to recognize it with a practical example.

What Is Simpson’s Paradox?

Simpson’s Paradox occurs when a relationship between variables appears in several separate groups but reverses or disappears when the groups are combined.

In simple terms:

The overall trend can be the opposite of the trend observed within individual groups.

This usually happens because a third variable, often called a confounding or grouping variable, affects the relationship between the variables being analyzed.

For example, suppose a company compares the success rates of two marketing campaigns.

Looking at all customers together might suggest that Campaign A performs better.

However, after separating customers by age group, you might discover that Campaign B performs better for both younger and older customers.

The aggregated result can therefore tell a completely different story.

A Simple Example

Imagine two students, Alice and Bob, taking exams at two difficulty levels.

We want to compare their success rates.

Easy Exams

Student	Passed	Total	Pass Rate
Alice	90	100	90%
Bob	19	20	95%

Bob performs better on easy exams.

Difficult Exams

Student	Passed	Total	Pass Rate
Alice	8	10	80%
Bob	72	100	72%

Alice performs better on difficult exams.

So far, the results are straightforward:

• Bob performs better on easy exams.
• Alice performs better on difficult exams.

But let’s create a different allocation of exams.

Suppose Alice takes mostly difficult exams while Bob takes mostly easy exams.

The aggregated results could make one student appear substantially better overall, even though the subgroup results tell a different story.

This illustrates why the distribution of observations across groups matters.

A Classic Numerical Example

Let’s use a clearer example involving two treatments.

Suppose a hospital is comparing Treatment A and Treatment B.

Patients are divided into two groups:

• Less severe cases
• More severe cases

Less Severe Cases

Treatment	Recovered	Total	Recovery Rate
A	90	100	90%
B	19	20	95%

Treatment B performs better:

B = 95%
A = 90%

More Severe Cases

Treatment	Recovered	Total	Recovery Rate
A	18	30	60%
B	6	10	60%

Here, the treatments perform equally.

Now suppose we change the numbers slightly so that Treatment A performs better in both groups.

Less Severe Cases

Treatment	Recovered	Total	Recovery Rate
A	90	100	90%
B	8	10	80%

More Severe Cases

Treatment	Recovered	Total	Recovery Rate
A	30	60	50%
B	36	80	45%

Treatment A performs better in both groups.

However, if the distribution of patients across severity levels is very different between the treatments, the overall result can potentially reverse.

That is the key mechanism behind Simpson’s Paradox.

How Aggregation Creates Simpson’s Paradox

The problem becomes easier to understand mathematically.

Suppose we calculate an overall success rate:

Overall Rate = Total Successful Outcomes / Total Observations

This combines observations from different groups.

But the groups may not have equal sizes.

For example:

Group 1 → 90% success
Group 2 → 50% success

If one treatment has most of its observations in Group 1 while another has most of its observations in Group 2, simply comparing the overall averages can produce a misleading conclusion.

The overall result is effectively a weighted average.

For two groups:

Overall Rate =
(Group 1 Rate × Group 1 Weight)
+
(Group 2 Rate × Group 2 Weight)

The weights depend on how many observations belong to each group.

This is why two datasets with identical subgroup rates can produce different overall rates if their group distributions differ.

A More Concrete Example

Suppose two sales representatives have the following results.

Small Customers

Salesperson	Sales	Customers	Conversion Rate
Alice	90	100	90%
Bob	8	10	80%

Alice performs better.

Large Customers

Salesperson	Sales	Customers	Conversion Rate
Alice	5	10	50%
Bob	45	100	45%

Alice also performs better.

Now calculate the overall rates.

Alice

Total Sales = 90 + 5 = 95
Total Customers = 100 + 10 = 110

Overall Rate = 95 / 110
             ≈ 86.4%

Bob

Total Sales = 8 + 45 = 53
Total Customers = 10 + 100 = 110

Overall Rate = 53 / 110
             ≈ 48.2%

Alice remains ahead.

But imagine the group distributions change dramatically.

Alice could receive mostly difficult customers while Bob receives mostly easy customers.

Even if Alice has a higher conversion rate within every customer segment, Bob could potentially have a higher overall conversion rate because of the different composition of their customer bases.

This is one of the most important lessons of Simpson’s Paradox:

Overall performance can depend heavily on who or what is being measured.

Why Simpson’s Paradox Matters in Data Science

Simpson’s Paradox is not simply a statistical curiosity.

It can affect real data science projects.

1. Business Analytics

Suppose a company compares conversion rates between two marketing campaigns.

Campaign A may have a higher overall conversion rate, but Campaign B could perform better within every customer segment.

Without segmentation, analysts might recommend the wrong campaign.

2. Healthcare Analytics

Treatment outcomes can depend on patient characteristics such as:

• Age
• Disease severity
• Risk level
• Previous treatment
• Gender
• Comorbidities

Aggregating all patients together can hide important differences.

3. Machine Learning

A model might appear to perform well overall while performing poorly for particular groups.

For example:

Overall accuracy = 94%

But after segmentation:

Group A accuracy = 98%
Group B accuracy = 75%

The overall metric hides the weaker performance of Group B.

4. A/B Testing

Suppose an experiment compares Version A and Version B.

The overall results might show:

Version A → 7.5% conversion
Version B → 7.2% conversion

You might conclude that Version A is better.

But after splitting users by device:

Desktop:
A → 8.0%
B → 7.5%

Mobile:
A → 4.0%
B → 5.0%

The interpretation becomes much more complicated.

The device distribution may be influencing the aggregate result.

Simpson’s Paradox vs Confounding

These concepts are closely related but should not be treated as exactly the same thing.

A confounding variable is a variable that is associated with both the predictor and outcome and can distort the apparent relationship between them.

Simpson’s Paradox describes a particular pattern in which the relationship can change or reverse after accounting for a third variable.

For example:

Campaign → Conversion

might appear to show that Campaign A is better.

But after introducing:

Campaign → Customer Segment → Conversion

the relationship may change.

The customer segment is therefore important for interpreting the original comparison.

How to Detect Simpson’s Paradox

There is no single statistical test that you can simply run to detect every instance of Simpson’s Paradox.

Instead, analysts should investigate whether aggregated relationships change after meaningful segmentation.

Step 1: Analyze the Overall Relationship

Start with the aggregated data.

For example:

Campaign A → 12% conversion
Campaign B → 10% conversion

Step 2: Identify Potential Grouping Variables

Look for variables that could influence the outcome.

Examples include:

• Age
• Location
• Device
• Customer type
• Income level
• Product category
• Experience
• Disease severity

Step 3: Segment the Data

Calculate the relevant metric separately for each group.

Segment 1 → A vs B
Segment 2 → A vs B
Segment 3 → A vs B

Step 4: Compare the Results

If the relationship changes substantially or reverses across the aggregated and segmented analyses, investigate further.

Simpson’s Paradox in Python

You can use Python and pandas to examine grouped results.

import pandas as pd

data = pd.DataFrame({
    "group": ["A", "A", "B", "B"],
    "method": ["X", "Y", "X", "Y"],
    "success": [90, 80, 50, 45],
    "total": [100, 100, 100, 100]
})

data["rate"] = data["success"] / data["total"]

print(data)

You can then calculate aggregate and group-level rates separately.

For larger datasets, groupby() is particularly useful:

grouped = (
    data.groupby(["group", "method"])
        .agg(
            success=("success", "sum"),
            total=("total", "sum")
        )
)

grouped["rate"] = grouped["success"] / grouped["total"]

print(grouped)

This makes it easier to compare performance across subgroups rather than relying exclusively on aggregate statistics.

How to Avoid Misleading Conclusions

The best defense against Simpson’s Paradox is not to avoid aggregation entirely.

Aggregation is useful.

The goal is to understand what is being aggregated.

Always Investigate Important Segments

If customer type, location, device, or another variable could influence your result, examine it.

Compare Group Sizes

Averages and rates can be heavily influenced by unequal group sizes.

Use Domain Knowledge

Statistical relationships should be interpreted in the context of the business or scientific problem.

Avoid Automatically Trusting Aggregate Metrics

An overall average is a summary, not necessarily the complete story.

Investigate Reversals

If a relationship changes after segmentation, don’t immediately choose one result over the other.

Ask why.

Simpson’s Paradox and Data Visualization

Visualization can make these patterns easier to spot.

An overall bar chart might show:

Method A → 70%
Method B → 65%

But a grouped visualization might reveal:

Group 1:
A → 80%
B → 75%

Group 2:
A → 55%
B → 50%

Or, in a true reversal, the group-level comparisons could point in the opposite direction from the overall comparison.

This is why data visualization is an important part of exploratory data analysis.

A single aggregate chart can hide the structure that becomes obvious after introducing a grouping variable.

Common Mistakes When Analyzing Aggregated Data

Looking Only at the Overall Average

An overall average may hide meaningful differences between groups.

Ignoring Sample Sizes

Two percentages cannot always be compared meaningfully without understanding how many observations produced them.

Treating Correlation as Causation

Simpson’s Paradox demonstrates how easily relationships can change when additional variables are considered.

Segmenting Without a Reason

Not every variable should be used to split the data.

Segmentation should be guided by statistical reasoning and domain knowledge.

Overlooking Data Collection Differences

Different groups may have been exposed to different conditions, populations, or selection processes.

Understanding how the data was generated is often just as important as analyzing the data itself.

Simpson’s Paradox in One Table

Concept	Explanation
Overall relationship	Relationship observed after combining groups
Group-level relationship	Relationship observed within individual groups
Third variable	Variable that helps explain why the relationship changes
Aggregation	Combining observations across groups
Reversal	Overall relationship differs from subgroup relationships
Main risk	Drawing incorrect conclusions from aggregate data

Simpson’s Paradox is a powerful reminder that the way you organize data can influence the conclusions you draw from it.

A relationship that appears strong in aggregated data may weaken, disappear, or even reverse after you account for another variable.

For data scientists and analysts, the practical lesson is simple:

Don’t stop at the overall number.

When you see an important relationship in your data, investigate the groups behind it.

Break down the data by relevant variables, compare sample sizes, visualize subgroup behavior, and use domain knowledge to understand what is happening.

The goal isn’t to avoid aggregate statistics. It is to understand when an aggregate statistic is hiding something important.

Frequently Asked Questions