Before accepting what a test result, a profile, or a striking detail seems to prove, ask how common the thing is in the first place, because evidence that sounds accurate can still be wrong most of the time when the underlying condition is rare.

How to do it

1. 1Notice the tell: a claim of accuracy ("99 percent accurate", "fits the profile", "the algorithm flagged it") with no mention of how rare the thing being detected is.
2. 2Get the base rate: out of 1,000 people (or transactions, or emails) like this one, how many actually have the condition?
3. 3Work it through in whole numbers. If 10 in 1,000 have it and the test catches 9 of them, but it also wrongly flags 5 percent of the other 990 (about 50 people), then roughly 9 of 59 positives are real: about 15 percent, not 99.
4. 4Ask for the false-positive rate as well as the hit rate. "Detects 90 percent of cases" says nothing about how many non-cases it also flags.
5. 5Update from the base rate rather than replacing it. Specific evidence should move you from the starting frequency, not make you forget it.
6. 6When the stakes are high and the result is positive, get an independent second test before acting. Two independent positives change the odds far more than one.

When to use it

• •A positive result from a screening test, a drug test, a background check, a plagiarism or AI-text detector, or a fraud flag.
• •A forensic or statistical "match" presented as proof of identity or guilt.
• •A vivid description that "sounds exactly like" a member of some rare category.
• •Security or policy arguments that a screening program will catch rare offenders in a very large population.

Counters

Evidence and how strong it is

Kahneman & Tversky (1973) showed that people given a personality sketch all but ignore the stated proportions of the groups the person was drawn from. Bar-Hillel (1980) mapped when base rates are neglected and argued the cause is perceived relevance: specific, causal-sounding information crowds out the general frequency. Eddy (1982) found that most physicians given a mammography problem estimated the chance of cancer after a positive result at around 75 percent when the correct answer was under 10 percent. Gigerenzer & Hoffrage (1995) showed the error shrinks sharply when the same problem is posed in natural frequencies (counts out of 1,000) rather than conditional probabilities, which is why the steps above use whole numbers. Evidence strength: strong (replicated experiments) for both the bias and the frequency-format remedy. Koehler (1996) cautions that the laboratory literature overstates how completely base rates are ignored in natural settings and that the appropriate base rate is often genuinely ambiguous.

1. Bar-Hillel, M. (1980). The Base-Rate Fallacy in Probability Judgments. Acta Psychologica, 44(3), 211-233
   The analysis of when and why base rates are neglected in favour of specific, seemingly more relevant information.
2. Kahneman, D., & Tversky, A. (1973). On the Psychology of Prediction. Psychological Review, 80(4), 237-251
   The original demonstrations that predictions track representativeness and ignore prior probabilities.
3. Eddy, D. M. (1982). Probabilistic Reasoning in Clinical Medicine: Problems and Opportunities. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under Uncertainty: Heuristics and Biases (pp. 249-267). Cambridge University Press
   The finding that most physicians greatly overestimated the probability of cancer given a positive mammogram.
4. Gigerenzer, G., & Hoffrage, U. (1995). How to Improve Bayesian Reasoning Without Instruction: Frequency Formats. Psychological Review, 102(4), 684-704
   Evidence that restating the problem in natural frequencies markedly improves correct inference.
5. Koehler, J. J. (1996). The Base Rate Fallacy Reconsidered: Descriptive, Normative, and Methodological Challenges. Behavioral and Brain Sciences, 19(1), 1-17
   The critique that base-rate neglect is less total outside the laboratory and that the correct base rate is often ambiguous.