LogicalMANIPULATIVE

Confounding Variable Omission

What it is

Presenting a correlation as evidence that one thing causes another while leaving out a third factor that drives both, so the audience accepts a causal story the data cannot support.

How it works

A confounder is a variable linked to both the supposed cause and the outcome without lying on the path between them. Ice-cream sales and drownings rise together because of summer; coffee drinking once appeared to cause lung cancer because coffee drinkers were more often smokers. Worked: suppose 60 percent of coffee drinkers smoke against 20 percent of abstainers, and smokers have ten times the lung-cancer rate; coffee drinkers will show more than double the cancer rate of non-drinkers although coffee does nothing. The lever is that a causal story is satisfying and the omitted variable is invisible — the audience cannot miss what is not on the slide. The most expensive case in medicine was hormone replacement: large observational studies found women on HRT had far less heart disease, the 2002 Women's Health Initiative trial found the opposite, and the difference was that women who chose HRT were healthier and wealthier to begin with. Politics runs on the same structure. The honest tools are randomization, adjustment for known confounders with the adjusted and unadjusted results both shown, and Hill's criteria; the manipulation is knowing the third variable and leaving it out because the raw correlation makes the better argument.

Real-world examples

  • •Observational studies through the 1990s reported roughly 40 to 50 percent less coronary heart disease among women using hormone replacement therapy; the randomized Women's Health Initiative trial, halted in 2002, found increased risk. The users had been healthier than non-users for reasons unrelated to the hormones.
  • •Messerli's 2012 note in the New England Journal of Medicine showed a strong correlation between a country's chocolate consumption and its number of Nobel laureates per capita — a deliberate illustration, since national wealth drives both.
  • •Claims that right-to-work states create more jobs, made by advocates of those laws, and claims that states with higher minimum wages have less poverty, made by advocates of those, both compare states that differ in region, industry mix, cost of living and wealth; each omits the variables that would deflate its correlation.
  • •Reinhart and Rogoff's 2010 association between high public debt and slow growth was widely read as debt causing slow growth; critics pointed out that slow growth also causes high debt, and the paper did not resolve the direction.
  • •Private schools report higher test scores than public schools; students whose families can pay tuition differ in income and parental education, and analyses that adjust for these find much smaller or no differences.

Ethical guidelines

  • ●When you present a correlation as causal, name the plausible third variables and say what you did about them; if the answer is nothing, do not use causal language.
  • ●Show unadjusted and adjusted results together so the reader can see what the adjustment did.
  • ●Prefer randomized or natural-experiment evidence for causal claims and say when the evidence is only observational.
  • ●Apply the same skepticism to correlations that flatter your side; the confounder you notice in the opponent's chart is usually in yours.

How to defend against it

  • ►For any “X is linked to Y”, ask what else differs between the people or places with more X and those with less, and whether that difference could produce Y on its own.
  • ►Ask whether the study was randomized; if not, ask what the authors adjusted for and what the result looked like before adjustment.
  • ►Ask about reverse causation: could Y be producing X instead? Debt and growth, wealth and health, and screen time and mood all run both ways.
  • ►Look for a randomized trial or a natural experiment on the same question before accepting the causal story; if none exists, hold the claim as a correlation.
  • ►When a comparison is across states, countries or schools, list the obvious ways the units differ before reading the correlation as an effect of the policy.

References

  1. Writing Group for the Women's Health Initiative Investigators (2002). Risks and benefits of estrogen plus progestin in healthy postmenopausal women: Principal results from the Women's Health Initiative randomized controlled trial. JAMA, 288(3), 321-333
    The randomized trial that reversed the observational finding of cardiovascular benefit from hormone replacement.
  2. Messerli, F. H. (2012). Chocolate consumption, cognitive function, and Nobel laureates. New England Journal of Medicine, 367(16), 1562-1564
    The chocolate-Nobel correlation used to illustrate confounding by national wealth.
  3. Hill, A. B. (1965). The environment and disease: Association or causation?. Proceedings of the Royal Society of Medicine, 58(5), 295-300
    The criteria for moving from an observed association to a causal claim.
  4. Pearl, J., & Mackenzie, D. (2018). The Book of Why: The New Science of Cause and Effect. Basic Books
    Accessible account of confounding, adjustment and why correlation alone cannot establish cause.
Last reviewed
Suggest a correction

Detect Confounding Variable Omission in any text

Paste any message, email, or article into our free Manipulation Detector to see if Confounding Variable Omission or other techniques are being used on you.

Try the Detector FreeCreate Free Account