“There’s too much secrecy, too little transparency and not enough accountability when it comes to these incredibly important statistics.”
Imagine you’re sitting on a jury. The prosecutor stands in front of you and says: “We found DNA at the crime scene that matches the defendant’s. In fact, there is only a one in 1 million chance that someone else’s DNA would be a match.”
It’s a compelling argument, seemingly built on science and eye-popping statistics. One in a million, you think. Pretty compelling.
But behind that stat is a whole set of fraught assumptions that complicate what it is to “match.” What kind of DNA was found? How complete was the sample? What’s the size of the database? One in a million sounds pretty impressive if you’re talking about, say, the residents of a town … but one in a million if your sample size is the world? That’s a lot of matches!
On this week’s episode of our podcast What’s The Point, Erin Murphy, a law professor at New York University and the author of the new book “Inside the Cell,” discusses the rise of forensic DNA and how we use science and statistics in our courtrooms.
To listen, stream or download the full episode above. Video and a partial transcript are below.
Also this week, FiveThirtyEight’s Andrew Flowers follows up on his previous significant digit about the popularity of various baby names, including Vivian (his new daughter’s name!) and Saint.
What the birthday problem can teach us about forensic evidence
Making the case with DNA
Jody Avirgan: The use of DNA to solve a crime ultimately manifests itself inside of a courtroom, as rhetoric, when you are making an argument using forensic evidence and statistics. So what are the challenges [when it comes to] making a rhetorical argument to a jury?
Erin Murphy: There are tremendous challenges. The things that juries find persuasive may not be persuasive to statisticians. A lawyer has to think about how to present the information ethically. If there’s something that’s true that a jury may like, but it’s not statistically founded, how far can [one] push that argument? Conversely, you may have a very good argument with the statistics, that is complicated and sophisticated, but you feel like a jury is never going to “get” it in a cross-examination format. Trying to balance how strategically you choose your arguments, how you present them, that is incredibly difficult.
Even something as simple as the match statistics in DNA. The match statistics are commonly reported as a “random match probability.” That’s the chance that if you pick a person at random that they will match this DNA sample.
But jurors don’t hear that. Jurors hear, “What’s the chance this guy’s guilty? What’s the chance this is his DNA?” They don’t hear the precise statistics. They hear general ideas about guilt or innocence.
Is DNA the best worst evidence we have?
Avirgan: [How do] you think about DNA evidence as compared to the other kinds of evidence that could be introduced? We know how unreliable eyewitness evidence can be. We know confessions can be coerced. Is DNA evidence still the most reliable in your mind?
Murphy: It’s impossible to answer in the abstract. It’s the same with [eyewitness] evidence. Different witnesses have their strengths and weaknesses. If you have a witness who had a long time to observe and wasn’t subject to pressures, you may have great confidence in that witness.
Same with DNA. If you have a sample from an intimate bodily swab, like in a rape case, or if you have a large quantity of sample, or if you’re able to recover a robust profile that doesn’t require a lot of subjective interpretation, that’s great evidence. We can rely on that.
But just because that form of DNA exists, it doesn’t mean that every form of DNA exists like that. There are weak versions. Just like there are the witnesses who didn’t have a chance to observe or are highly suggestible. You can have DNA that is in a low quantity or was put in a database that could lead to a fortuitous match instead of actually being indicative of a true source.
It’s really just that we need to understand that DNA is just as complicated. It’s not going to be our shining savior.
If you’re a fan of What’s The Point, subscribe on Apple Podcasts, and please leave a rating/review — that helps spread the word to other listeners. And be sure to check out our sports show Hot Takedown as well. Have something to say about this episode, or have an idea for a future show? Get in touch by email, on Twitter, or in the comments.