LogicalMANIPULATIVE

Prosecutor's Fallacy

What it is

Presenting the probability of the evidence given innocence — “only one person in a million would match” — as if it were the probability of innocence given the evidence, so that a rare coincidence is heard as near-certain guilt.

How it works

Two conditional probabilities are transposed. The chance that an innocent person would match the evidence is one thing; the chance that a person who matches is innocent is another, and the second depends on how many people could have matched. Worked: a forensic match with a one-in-a-million random-match probability, in a country of 300 million, is shared by about 300 innocent people; with no other evidence, the suspect is one of roughly 301 and the probability of guilt is a fraction of a percent, not 99.9999 percent. Thompson and Schumann named the error in 1987 and showed mock jurors committing it. The lever is that ordinary language does not distinguish the two probabilities — “the chance this is a coincidence is one in a million” means either — and the tiny number is vivid while the base of possible matches is invisible. The error compounds when the small number was itself built by multiplying probabilities that were not independent. Roy Meadow's testimony at Sally Clark's 1999 trial squared a 1-in-8,543 cot-death figure to get 1 in 73 million, ignoring that risk factors cluster in families, and then let the jury hear the result as the odds of her innocence. The Royal Statistical Society said publicly in 2001 that the figure had no statistical basis.

Real-world examples

  • •Sally Clark was convicted in 1999 of murdering two infant sons after evidence that two cot deaths in a family like hers had a chance of 1 in 73 million; the Royal Statistical Society issued a statement in October 2001 objecting to the figure, and her conviction was quashed in January 2003. Hill (2004) later estimated that, given two infant deaths, a double natural death was several times more likely than a double murder.
  • •In People v. Collins (1968) a California prosecutor multiplied the estimated frequencies of a couple's characteristics — a blonde woman with a ponytail, a Black man with a beard, a yellow car — to arrive at 1 in 12 million; the state supreme court reversed the conviction, noting both the invented probabilities and the failure to ask how many matching couples existed.
  • •Lucia de Berk, a Dutch nurse, was convicted in 2003 after evidence that the chance of her presence at so many deaths was 1 in 342 million; statisticians showed the calculation was built on selected incidents and transposed conditionals, and she was exonerated in 2010.
  • •DNA database searches produce “cold hits” by comparing a crime-scene profile against millions of stored profiles; a match probability of one in a million means several innocent matches are expected in a database of that size, a point that prosecutors have sometimes presented as if the match alone were conclusive.

Historical case studies

R v Sally Clark and the "1 in 73 million" figure

1999–2003English Criminal Law

Solicitor Sally Clark was convicted in 1999 of murdering her two infant sons after paediatrician Sir Roy Meadow told the jury that the chance of two cot deaths in a family like hers was 1 in 73 million. He had squared the single-death figure, treating the deaths as independent when genetic and environmental factors make a second death more likely, and the number was easily heard as the probability that she was innocent, which it was not. The Royal Statistical Society publicly criticized the evidence in 2001. The Court of Appeal quashed the convictions in 2003, chiefly because a pathologist had withheld test results pointing to infection, and added that the statistic should never have gone before the jury.

Source →

People v. Collins

1968California Supreme Court

A couple was convicted of a Los Angeles robbery after the prosecutor called a mathematics instructor, assigned invented probabilities to six traits the witnesses had described (a blonde ponytail, a bearded Black man, a yellow car and so on), multiplied them, and told the jury there was one chance in twelve million that any couple other than the defendants had committed the crime. The California Supreme Court reversed. The figures had no evidentiary basis, the traits were not independent, and even a correct figure would describe how rare such couples are, not the likelihood that this couple was guilty. The opinion called mathematics "a veritable sorcerer in our computerized society."

Source →

Ethical guidelines

  • ●When presenting a match probability, state the number of people who could have matched and let the jury or audience see the resulting odds, not just the small number.
  • ●Never multiply probabilities of characteristics without evidence that they are independent, and never present the product as the odds of innocence.
  • ●Say explicitly which conditional you are giving: the chance of this evidence if the person is innocent, not the chance the person is innocent.
  • ●Expert witnesses should decline to convert an evidential probability into a verdict probability; that step requires the prior and is the tribunal's to take.

How to defend against it

  • ►When you hear “only one in N would match”, multiply by the size of the relevant population; that is the number of innocent matches you are being asked to ignore.
  • ►Ask which way round the probability runs: the chance of the evidence if innocent, or the chance of innocence given the evidence? Only the second is the verdict, and it needs the base rate.
  • ►Ask how the small number was built; if characteristics or events were multiplied together, ask what evidence there is that they were independent.
  • ►Ask what other evidence points to this suspect rather than to anyone else who matches; a match narrows the field, it does not choose within it.
  • ►Read the Royal Statistical Society's Sally Clark statement or the Collins judgment: both are short and both show the error in a form a juror can follow.

From the Defense Playbook

The Base-Rate Checkminutes

Before accepting what a test result, a profile, or a striking detail seems to prove, ask how common the thing is in the first place, because evidence that sounds accurate can still be wrong most of the time when the underlying condition is rare.

Every playbook entry states how strong its evidence is and when not to use it. Browse the full playbook.

References

  1. Thompson, W. C., & Schumann, E. L. (1987). Interpretation of statistical evidence in criminal trials: The prosecutor's fallacy and the defense attorney's fallacy. Law and Human Behavior, 11(3), 167-187
    The naming and the mock-juror demonstration of the transposed-conditional error.
  2. Royal Statistical Society (2001). Royal Statistical Society concerned by issues raised in Sally Clark case. Press statement, 23 October 2001
    The Society's public objection to the 1-in-73-million figure and its use at trial.
  3. Hill, R. (2004). Multiple sudden infant deaths — coincidence or beyond coincidence?. Paediatric and Perinatal Epidemiology, 18(5), 320-326
    The comparison of double natural death against double murder, given two deaths, in the Clark case.
  4. Gigerenzer, G. (2002). Calculated Risks: How to Know When Numbers Deceive You. Simon & Schuster
    Natural-frequency presentation of DNA and match evidence that makes the number of innocent matches visible.
Last reviewed
Suggest a correction

Detect Prosecutor's Fallacy in any text

Paste any message, email, or article into our free Manipulation Detector to see if Prosecutor's Fallacy or other techniques are being used on you.

Try the Detector FreeCreate Free Account