Prosecutor’s Fallacy

Following on from my previous post, I will do some ranting and raving about possibly the most common and insidious statistical mistake. Namely: P(A|B) = P(B|A). A less abstract expression of this equality lies at the heart of the “Prosecutor’s Fallacy”.

Here is the setup:

• The defendant’s DNA matches that found at the crime scene; a match obtained from a search of a DNA registry

• The probability of a match between the DNA at the scene and a random person is 1 in a million (i.e. false positives is 1/million)

• The defendant maintains they are innocent of the charges

Thus: “Because the story before the court is highly improbable, the defendant’s innocence must be equally improbable.” The prosecutor is effectively arguing: “the likelihood of this DNA being from any other person than the defendant is 1 in a million”.

If we let “positive DNA match” = M and “guilt” = G then what is in question is the probability of guilt given a match: P(G|M). The prosecutor's argument is that because the chance of an erroneous match is small P(M|not G)=1/million [or alternatively, the probability of a match given that you are guilty is large P(M|G)=million-1/million] thus the probability of being guilty given a match is large. Hence the prosecutor is arguing that P(G|M)=P(M|G).

Now being well versed in Bayes’ theorem we know this cannot be the whole story. Because P(G|M) = P(M|G)P(G)/P(M).

What the prosecutor is arguing is equivalent to claiming that: the probability that you speak English, given that you are Australian, is equal to the probability that you are Australian, given that you speak English. Which is absurd.

If one assumes that the defendants prior probability of guilt is 1 in a million i.e. P(G) = 1/million and that the probability of a match given that the defendant is guilty is 1 i.e. P(M|G)=1 and work through the math, it turns out P(G|M) = 0.5. Not really “beyond reasonable doubt”. Note however, that if the prior is increased to 1/10,000 [perhaps the defendant went to school with the victim] then P(G|M) = 0.99.

So, how does one quantify P(G)?

If you’re interested there is an equally fascinating “defense attorney’s fallacy”; it is possibly the reason O. J. Simpson is a free man.

-------------------------

Adapted from MATH3104 lecture notes by Prof Geoff Goodhill