LogicalDUAL-USE

Misleading Averages

What it is

Reporting a single “average” — a mean, median or mode chosen for effect, or a mean of a skewed, lumpy or oddly weighted distribution — so that the number describes almost nobody while the audience assumes it describes the typical case.

How it works

The word “average” covers three different numbers, and for a skewed distribution they are far apart. Ten workers, nine paid 40,000 and one paid a million: the mean is 136,000, the median is 40,000, and a presenter chooses between them by what they want the audience to conclude about pay. Huff called it the well-chosen average. Incomes, wealth, house prices, damages awards, hospital stays and investment returns are all skewed, so the mean sits above most people. Averages also fail on their own terms when the thing they feed into is nonlinear: Savage's flaw of averages is that a plan built on the average demand, the average delay or the average return will systematically underperform, because shortfalls and overshoots do not cancel. Fund marketing exploits a special case: a fund that gains 50 percent one year and loses 50 percent the next has an arithmetic average return of zero while the investor has lost a quarter of their money. Weighting is the quietest lever — averaging countries, years or firms with equal weight regardless of size can flip a result. The honest use shows the distribution or names which average and why.

Real-world examples

  • •Debate over the 2017 US Tax Cuts and Jobs Act ran on two averages from the same Tax Policy Center analysis: supporters cited the mean tax cut across all households, pulled up by large cuts at the top, while opponents cited the smaller cut for the median household. Both were computed correctly from the same law.
  • •Reinhart and Rogoff's 2010 finding that growth collapses when public debt passes 90 percent of GDP rested partly on averaging each country's average rather than each country-year, so that one year of New Zealand data weighed the same as nineteen years of British data; Herndon, Ash and Pollin (2014) showed that this choice, a spreadsheet error and some exclusions turned a slightly negative growth figure into a positive one.
  • •The statement that life expectancy in 1900 was about 47 years is a mean dragged down by infant and child mortality; a person who reached adulthood then could expect to live into their sixties. The average is used to imply that middle age was old age.
  • •Fund literature that reports “average annual return” as an arithmetic mean of yearly returns overstates what a buy-and-hold investor received whenever returns are volatile, because losses and gains of equal size do not cancel.

Ethical guidelines

Where the line is

An average is legitimate when the presenter names which one it is, shows the median or the distribution beside a mean of skewed data, and states the weighting; it becomes manipulation when the type of average, the weighting or the omitted spread is chosen because the audience will picture a typical case that the number does not describe.

  • ●Say which average you are using and why, and give the median beside the mean whenever the distribution is skewed.
  • ●Show the spread — a range, quartiles or the distribution itself — whenever the audience will use the number to picture a typical case.
  • ●State the weighting scheme when averaging across units of different sizes, and report the result under the obvious alternative.
  • ●For returns, report the compound (geometric) figure an investor would actually have earned, not the arithmetic mean of annual returns.

How to defend against it

  • ►Ask “which average?” and ask for the median when you hear a mean, and the mean when you hear a median; a large gap between them tells you the distribution is lopsided and the typical case is not the average.
  • ►Ask what the distribution looks like: how many are below, how many far above, whether there are two clumps. A single number cannot answer, and a source that has the data can.
  • ►For anything averaged across countries, firms or years, ask how the units were weighted and whether the conclusion survives weighting by size.
  • ►For investment returns, ask for the growth of a fixed sum over the period rather than the average yearly return.
  • ►When an average is used to plan — average demand, average commute, average patient — ask what happens on the bad side of the average, since that is where the cost is.

From the Defense Playbook

The Unit-Price Checkseconds

Compare products by price per unit (per 100 grams, per litre, per sheet, per dose) instead of by pack price, pack size, or the size of the discount sticker, which is the one number that package design and promotions cannot disguise.

Decide Metrics Before Testingminutes

Before an A/B test or pilot starts, write down the primary metric, the harm metrics that must not get worse, the sample size or stopping rule, and the planned analysis, so that the result can tell you something you did not already want to hear.

Every playbook entry states how strong its evidence is and when not to use it. Browse the full playbook.

References

  1. Huff, D. (1954). How to Lie with Statistics. W. W. Norton
    The chapter on the well-chosen average: mean, median and mode selected for effect.
  2. Savage, S. L. (2009). The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty. Wiley
    Plans built on average inputs systematically fail when outcomes are nonlinear.
  3. Herndon, T., Ash, M., & Pollin, R. (2014). Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff. Cambridge Journal of Economics, 38(2), 257-279
    The country-weighting, exclusion and spreadsheet issues in the 90 percent debt-threshold result.
  4. Reinhart, C. M., & Rogoff, K. S. (2010). Growth in a time of debt. American Economic Review: Papers and Proceedings, 100(2), 573-578
    The original paper whose averaging scheme was at issue.
Last reviewed
Suggest a correction

Detect Misleading Averages in any text

Paste any message, email, or article into our free Manipulation Detector to see if Misleading Averages or other techniques are being used on you.

Try the Detector FreeCreate Free Account