Imagine that one treatment appears more successful than another when patients are separated into two groups, yet looks less successful when all patients are combined. The arithmetic can be correct in both cases. This surprising result is known as Simpson’s paradox. It shows how a summary statistic can conceal important differences between groups and why understanding the data behind an average matters as much as calculating the average itself.
Key takeaways
- Simpson’s paradox occurs when a relationship changes or reverses after data are combined or separated into groups.
- Different group sizes can make overall averages misleading.
- The phenomenon is mathematically possible without any calculation error.
- A third variable may help explain why subgroup and overall comparisons differ.
- Statistical association alone does not establish which action causes a better outcome.
What is Simpson’s paradox?
Simpson’s paradox is a statistical phenomenon in which an association seen in aggregated data disappears or reverses when observations are divided into relevant subgroups, or vice versa. The name honours statistician Edward H. Simpson, whose 1951 paper discussed related patterns, although earlier researchers had examined similar problems.
The word paradox does not mean mathematics has contradicted itself. The calculations use different sets of observations and different weights. What feels surprising is that people often assume a comparison that holds in every subgroup must also hold when those groups are combined. That assumption is not always true.
A simple example with two schools
Imagine two fictional schools, A and B, each offering an advanced course and a foundation course. In the advanced course, School A passes 18 of 20 pupils, or 90%, while School B passes 72 of 90, or 80%. In the foundation course, A passes 36 of 60, or 60%, while B passes 5 of 10, or 50%. School A has the higher pass rate in both courses.
Now combine the results. School A passes 54 of 80 pupils, or 67.5%. School B passes 77 of 100, or 77%. The overall ranking has reversed: B appears better even though A has the higher rate in each course. These numbers are hypothetical and designed to demonstrate the mathematics, not to describe real schools.
Check the arithmetic
For School A, the advanced course contributes 18 passes and the foundation course contributes 36, giving 54 passes among 80 pupils. For School B, 72 plus 5 gives 77 passes among 100 pupils. There is no error in addition or division. The difference lies in how many pupils each school has in the easier and harder course.
School B has a much larger share of pupils in the advanced course, where pass rates are high for both schools. School A has more pupils in the foundation course, where pass rates are lower. The combined averages therefore reflect course mix as well as the differences between schools.
Why weighted averages matter
An overall rate is a weighted average of subgroup rates. A subgroup containing ninety pupils contributes more to the combined result than a subgroup containing ten. The weights are the proportions of the total population in each group. If two institutions have different group proportions, their overall averages may differ even when one performs better in every comparable subgroup.
A common mistake is to take the average of two percentages without considering how many observations each percentage represents. In the school example, the two courses do not have equal numbers of pupils. Calculating a simple average of the course percentages would answer a different question from calculating the overall pass rate.
What is a confounding variable?
A confounding variable is a factor related to both the explanatory variable and the outcome that can distort a comparison. In the school example, course type affects the difficulty of achieving a pass and differs between the schools. Ignoring it mixes together pupils facing different circumstances.
This does not prove that course type is the only explanation. Other factors such as prior attainment, teaching resources and selection policies may matter. The important point is that an apparently straightforward comparison can conceal a relevant difference in the groups being compared.
Association is not the same as causation
Suppose School B has the higher overall pass rate. It would be a mistake to conclude immediately that attending B causes better results. The difference might reflect which pupils enrol, which courses they take and how pass rates are measured. Earth Site’s guide to correlation versus causation explains why an observed relationship is not automatically evidence of a cause.
Simpson’s paradox is particularly important in causal reasoning because the combined and separated results can appear to support opposite conclusions. Deciding which comparison is appropriate requires understanding how the groups were formed and what question is being asked.
A medical example
Medical treatments may be used for patients with different levels of illness. A treatment given mainly to severely ill patients could have a lower overall recovery rate than another treatment used mostly for mild cases, even if it performs better within both severity groups. The aggregate result can reflect the mix of cases rather than the treatment’s effectiveness.
This is a hypothetical illustration, not medical advice. Real clinical decisions require properly designed evidence, attention to side effects and assessment of individual circumstances. Simpson’s paradox teaches why comparisons should consider important patient characteristics instead of relying on one overall percentage.
Why grouping can change the result
Combining data removes information about the groups to which observations belong. If those groups have different baseline rates, the resulting average may reflect group composition. Separating data can reveal a pattern hidden in the total, but grouping can also create problems if it is done arbitrarily or based on a variable affected by the outcome.
The solution is not simply ‘always divide data into more groups’. Too many small groups can produce unstable estimates, and choosing groups after seeing the results can encourage selective reporting. Researchers need a reasoned approach based on the process that generated the data.
Simpson’s paradox in university admissions
A well-known historical example concerns admissions data from the University of California, Berkeley, in the 1970s. Overall admission rates differed between male and female applicants. Examination by department revealed that application patterns across departments with different selectivity were important to understanding the aggregate difference.
The case is often used to demonstrate why an overall disparity cannot be interpreted without context. It should not be reduced to the claim that discrimination is always absent whenever group composition explains a statistical pattern. Assessing fairness requires investigating procedures, opportunities and potential barriers as well as the numbers.
How to spot the paradox in a table
Start with the overall rates for two groups. Then identify a third variable that could plausibly affect the outcome, such as age, department, disease severity or region. Calculate the rates separately within its categories. If the direction of the association changes, examine the numbers and weights that produce the reversal.
Always record the numerator and denominator. A percentage without the number of observations can hide the role of sample size. Earth Site’s explanation of sampling bias adds another warning: even carefully calculated percentages can mislead if the observations do not represent the population of interest.
How to spot it in a graph
A scatter graph may show a trend across all observations while differently coloured subgroups each show another trend. For example, a combined set of points might slope downwards while each subgroup slopes upwards. The apparent contradiction can result from the groups occupying different ranges of the horizontal and vertical axes.
Visualisation helps reveal the structure, but it does not identify a cause automatically. A graph should show units, sample sizes where possible and the meaning of each group. Without that information, viewers may mistake a mathematical pattern for a physical explanation.
What does standardisation do?
Standardisation can compare groups using a common set of weights. For example, researchers comparing mortality rates between countries may adjust for differences in age structure. An older population can have a higher crude death rate even if age-specific mortality is lower at many ages.
Age-standardised rates answer a different question from crude rates: what would the comparison look like under a specified common age distribution? Neither statistic is universally ‘the correct one’. The choice depends on whether the aim is to describe actual deaths or compare risks under similar demographic conditions.
Why population structure matters
Countries, cities and schools differ in their populations. If a statistic depends strongly on age, income, occupation or course difficulty, a change in group composition can shift the overall average without a corresponding change within each group. This is why demographic information matters in public health, economics and education.
Earth Site’s guide to how population censuses work explains the collection of population data used for many such comparisons. Reliable denominators and clear group definitions are essential. Without them, a rate can look precise while describing the wrong population.
The difference between description and decision
Sometimes the overall rate is exactly what a decision-maker needs. A hospital manager may need to know how many patients recovered across the entire hospital to plan services. But a patient deciding between treatments may need outcomes among people with similar medical conditions. The relevant statistic depends on the question.
This is a central lesson of Simpson’s paradox. An aggregate comparison is not automatically false, and a subgroup comparison is not automatically causal. Each describes a particular distribution of observations. Good reasoning begins by stating the decision or explanation the analysis is intended to support.
A step-by-step classroom activity
Give students the fictional school figures from the earlier example. Ask them to calculate each course pass rate and the combined pass rate. Next, have them explain in words why the school ranking changes. Encourage them to draw two tables, one showing percentages and another showing pupil counts.
Then ask a harder question: would the result change if the schools enrolled equal proportions of pupils in the two courses? Students can apply common weights and see how the comparison changes. The activity turns an apparently mysterious paradox into a lesson about denominators, composition and interpretation.
What researchers do to avoid mistakes
Researchers plan relevant subgroup analyses, collect information about potential confounders and use statistical models appropriate to their questions. Random assignment in experiments can reduce systematic differences between groups, although it does not guarantee that every sample will be perfectly balanced. Observational studies need additional assumptions and careful comparison.
Transparency matters. A report should show how groups were defined, which observations were excluded and how uncertainty was estimated. If researchers searched many possible subgroup divisions, readers need to know that too. A dramatic reversal discovered after extensive exploration may require confirmation in independent data.
Common misconceptions
One misconception is that Simpson’s paradox proves statistics cannot be trusted. In fact, it demonstrates the need to understand how statistics are constructed. Another is that the smaller subgroup figures are always more meaningful. They may be, but only when the grouping variable is relevant to the question.
A third misconception is that a reversal must be caused by dishonesty. The pattern can arise naturally from differences in group composition. Misleading presentation may still occur when someone knowingly selects the comparison that supports a preferred story, but the mathematics itself is not evidence of deception.
How to read a headline using this lesson
When a headline compares schools, hospitals, regions or countries, ask what the percentages are based on. Could the groups contain different mixes of people or cases? Are the numerators and denominators shown? Have relevant categories been combined? Is the claim descriptive, or does it assert that one factor caused the result?
A useful response is not to reject the headline automatically. Instead, look for the underlying table, compare like with like and consider whether the summary answers the intended question. That habit improves statistical literacy far beyond this one paradox.
Conclusion
Simpson’s paradox reveals how correct calculations can support surprising and apparently conflicting summaries. The key lies in weighted averages, subgroup composition and the distinction between description and causation. By checking denominators, examining relevant groups and asking what question a statistic answers, students can avoid some of the most common mistakes in interpreting data.
Frequently Asked Questions
Is Simpson’s paradox a mathematical error?
No. Both the subgroup and overall calculations can be correct because they use different weights.
Why do the results reverse?
Different proportions of observations in subgroups with different baseline rates can change the combined comparison.
Does the paradox prove discrimination or unfairness?
No. It can reveal a misleading summary, but assessing discrimination requires additional evidence about causes and processes.
Is the overall average useless?
No. It may answer a legitimate question about the total population, even when subgroup comparisons answer a different question.
How can I check for Simpson’s paradox?
Calculate rates with their denominators, divide data into relevant groups and compare the subgroup patterns with the aggregate result.
