LogicalMANIPULATIVE

Ecological Fallacy

What it is

Inferring what is true of individuals from statistics about the groups or places they belong to — a state, a county, a neighborhood — when the individual-level relationship can be weaker, absent or reversed.

How it works

Robinson's 1950 paper made the problem famous. Across US states in 1930, the share of residents born abroad was positively correlated with literacy: more immigrants, more literate. Among individuals, the foreign-born were less literate than the native-born. Immigrants had settled in states whose native populations were already highly literate, and the state-level number described where they lived, not what they could do. Group statistics mix composition with behavior, and an aggregate correlation can be produced entirely by people who are not the ones being blamed or credited. Worked: county A is 60 percent X-voters and has twice county B's crime rate; B is 40 percent X-voters. Nothing in those two numbers says whether a single X-voter committed a crime — it is consistent with all the crime being committed by non-X residents of A. Gelman's red-state paradox is the political version: richer states vote for Democrats while richer individuals, especially in poorer states, vote for Republicans. The persuasion move is the sentence “counties that voted for X have more Y”, followed by a claim about X voters. Only individual-level data can support the second claim, and the presenter who has only the first knows it.

Real-world examples

  • •Robinson (1950) reported a state-level correlation of about plus 0.5 between foreign birth and literacy in the 1930 census, against an individual-level correlation of about minus 0.1: the aggregate pattern was the opposite of the individual one.
  • •Gelman and colleagues (2008) documented that the wealthiest states lean Democratic while wealthier individuals lean Republican, a reversal that commentators on both sides had used to make claims about “rich voters” from state maps.
  • •County-level analyses in late 2021 showing higher COVID-19 death rates in counties that had voted for Trump were widely repeated as statements about Trump voters' individual choices; the county figures were consistent with that story but could not establish it, since counties differ in age, health, rurality and hospital access.
  • •Claims that “Democrat-run cities” have the most murders argue from a city's aggregate to the conduct or policies of its residents and officials; the same aggregate is consistent with crime concentrated in places and populations that the city's politics did not create.
  • •Durkheim's 1897 finding that Protestant regions of Europe had higher suicide rates than Catholic ones has been re-read for a century as a statement about Protestant individuals, which regional data cannot show.

Ethical guidelines

  • ●Keep the unit of analysis in the sentence: “counties with more X have more Y” is not “people who are X do Y”, and the second requires individual data.
  • ●When only aggregate data exist, say what individual-level patterns they are and are not consistent with.
  • ●Do not use a map or a regional correlation to characterize the people in the region, in either direction.
  • ●When the individual-level data exist and contradict the aggregate story, report them.

How to defend against it

  • ►Ask what the unit of the data is — people, or places — and whether the claim being made is about the same unit.
  • ►Ask whether any individual-level data (surveys, records, exit polls) support the same conclusion; if the only evidence is a map, the conclusion about people is a guess.
  • ►Construct the alternative story: could the aggregate pattern be produced entirely by people other than the ones being blamed? If yes, the data do not distinguish.
  • ►Treat “places that did X have more Y” claims from your own side with the same suspicion; the fallacy is symmetric and both sides use it.

References

  1. Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15(3), 351-357
    The 1930 census analysis showing opposite signs for state-level and individual-level correlations of foreign birth and literacy.
  2. Gelman, A., Park, D., Shor, B., Bafumi, J., & Cortina, J. (2008). Red State, Blue State, Rich State, Poor State: Why Americans Vote the Way They Do. Princeton University Press
    The reversal between state-level and individual-level relationships of income and party vote.
  3. King, G. (1997). A Solution to the Ecological Inference Problem: Reconstructing Individual Behavior from Aggregate Data. Princeton University Press
    The formal statement of what aggregate data can and cannot reveal about individuals, and the methods required to say anything.
Last reviewed
Suggest a correction

Detect Ecological Fallacy in any text

Paste any message, email, or article into our free Manipulation Detector to see if Ecological Fallacy or other techniques are being used on you.

Try the Detector FreeCreate Free Account