The authors of “Justices Flunk Math” worry that math—or more narrowly, statistics and probabilities—is being misused in the courtroom. After looking at a few examples, they conclude:
The challenge is to make sure that the math behind the legal reasoning is fundamentally sound. Good math can help reveal the truth. But in inexperienced hands, math can become a weapon that impedes justice and destroys innocent lives.
Inexperienced and experienced hands use math as a weapon. Let’s take, for example, the analogy they use in the article.
The appellate judge in the Amanda Knox case refused to retest the murder weapon for traces of the victim’s DNA. According to the article, the judge claimed that if there was too little material to provide a reliable result the first time it was tested, in 2007, then tests on less material in 2011 would be no better.
Although the authors claim that the judge “demostrated a clear mathematical fallacy,” they describe a simple flaw in reasoning and then use a misleading mathematical example to make their case. They are right. Repeated tests of the same material in comparable conditions can confirm or disconfirm previous results. And the greater number of confirming or disconfirming results should influence the conclusions we draw from those results. But their analogy, while superficially persuasive, doesn’t necessarily apply to the judge’s decision not to perform additional DNA tests.
Imagine, for example, that you toss a coin and it lands on heads 8 or 9 times out of 10. You might suspect that the coin is biased. Now, suppose you then toss it another 10 times and again get 8 or 9 heads. Wouldn’t that add a lot to your conviction that something’s wrong with the coin? It should.
According to the article, the judge’s decision was based on there being less material to test. How much less? We are not told. But less. So a better analogy might be (my changes are in bold):
Imagine, for example, you toss a coin and it lands on heads 8 or 9 times out of 10. You might suspect that the coin is biased. Now, suppose you then toss it another 3 times and get 2 heads. Would that add a lot to your conviction that something’s wrong with the coin? Maybe.
We can play with the numbers for that second coin-toss series, but a useful analogy here is not 10 tosses followed by another 10 but followed by some smaller number. How much the second coin-toss series will add to our conviction that the coin is biased will depend on how many times we toss it and how many times it comes up heads.
At first glance their math looks persuasive. The information they present in the article, however, does not allow us to assess the relevance of their analogy or evaluate the judge’s decision. We are being asked, on the strength of their mathematical analogy, to accept their criticism of the judge. In this case, their math distracts us from asking about why the judge made his decision and whether or not that was a reasonable decision. But this is nothing new. People hoping to achieve certain ends apply the math they think will be most persuasive. Math is always a weapon.