All Episodes
Previous Next
Plain Text
March 1, 2022 - Skeptoid
19:01
Skeptoid #821: Forensic (Pseudo) Science

All those fancy-sounding forensic sciences you've come to trust might not be worth very much of your trust after all. Learn about your ad choices: dovetail.prx.org/ad-choices

Transcriber: nvidia/parakeet-tdt-0.6b-v2, sat-12l-sm, and large-v3-turbo
|

Time Text
Courtroom Science vs Legal Findings 00:08:05
When you turn on the television and settle in to watch your favorite crime drama, you know you're going to hear about ballistics tests on bullets, fingerprints, DNA matching, and blood spatters.
All the familiar forensic evidence techniques certainly sound science-y, and they're on television, so they must have legitimate science foundations, right?
Well, we're going to find out today on Skeptoid.
Hi, I'm Alex Goldman.
You may know me as the host of Reply All, but I'm done with that.
I'm doing something else now.
I've started a new podcast called Hyperfixed.
On every episode of Hyperfixed, listeners write in with their problems and I try to solve them.
Some massive and life-altering, and some so minuscule it'll boggle your mind.
No matter the problem, no matter the size, I'm here for you.
That's HyperFixed, the new podcast for Radiotopia.
Find it wherever you listen to podcasts or at hyperfixedpod.com.
You're listening to Skeptoid.
I'm Brian Dunning from Skeptoid.com.
Forensic Pseudoscience Something we've pointed out on Skeptoid many times over the years is that science is not done in the courtroom.
People are constantly mistaking legal findings for scientific findings.
They are not.
Judges and juries can be persuaded of things by any number of methods, and solid science is rarely one of those.
Even when good science is presented in the courtroom, lawyers are free to argue over its validity, thus wrongly creating the impression that the science itself is shaky.
But just as harmful is the opposite process, the introduction of bad science in a courtroom, alongside a deep-rooted cultural belief that the science is a valid one.
And here we are talking about forensic science, the bits of evidence that link a killer to a crime scene, believed by many to be irrefutable.
Today, we're going to check in on the current status of the tidal wave of reform that should be crashing down upon forensic pseudoscience, but which has, oddly, stalled offshore somewhere and is nowhere to be found.
In 1987, the very first criminal conviction based on DNA evidence was secured, that of a Florida rapist.
Then in 1989, the first exoneration of a previously convicted man due to DNA evidence took place.
This blossomed throughout the 1990s, as DNA was used more and more often to prove with scientific certainty that convictions had been falsely obtained, convictions which were based on other types of evidence.
Most of these overturned convictions had been obtained based on eyewitness misidentification, false or coerced confessions, or through the testimony of informants.
But of all of these, half also involved false or misleading forensic evidence.
The Wonderful Innocence Project, a U.S.-based nonprofit legal organization that works to overturn improper criminal convictions, maintains a white paper breaking all this down.
Among the most offending junk sciences they point out are hair analysis, bite mark analysis, shoe print comparisons, bullet lead analysis, and certain types of evidence from a fire allegedly indicating arson.
Sadly, these are just the tip of the iceberg.
But wait a minute, we're tempted to respond.
I've been watching all my favorite cop investigation dramas on television for decades, and I know all these things are rock solid scientifically.
Well, it turns out we may believe that, but it's in surprisingly sharp contradiction with the facts.
Because when all these convictions started getting overturned due to DNA evidence, prosecutors started getting upset and nervous.
Advocates for criminal justice reform started getting louder and bolder.
Courts and lawmakers started taking notice.
And what has followed so far has been a giant wave of nothing.
The status quo of bad and misrepresented science still reigns.
The basic root of the problem is that prosecutors do not want scientists in the courtroom.
They're called prosecutors because their job is to convict.
It's not to investigate, not to learn the truth, not to bring about justice.
Those are the domains of investigators and judges and others in the chain.
Prosecutors want the tools of the courtroom to be those that result in convictions.
So the majority of these forensic evidence techniques were not developed by scientists.
They were developed by prosecutors and by law enforcement.
Therefore, we should expect that such forensic techniques would have minimal, if any, scientific basis.
And at every level of our criminal justice system, it is the prosecutors, from local to federal, who are primarily responsible for introducing evidence into our courtrooms.
Given this structure, the surprise would be if the evidence in our courts did pass scientific scrutiny.
All of this new disgrace that DNA analysis had brought to the other types of forensic evidence prompted Congress to pass a new law in 2005, which in part directed the National Academy of Sciences to conduct a study on forensic science.
They were to make an assessment and then create recommendations and best practice guidelines.
The result of this study came in 2009 with a 300-plus page book, Strengthening Forensic Science in the United States, A Path Forward.
Freely available online as a PDF.
This book goes into great detail on just about every forensic evidence discipline you've heard of, and it generally finds that the scientific validity of each varies widely from one to another.
Although this was nice to have, unfortunately, little was done with it, and innocent people continued to be convicted based on pseudoscience.
And this was when politics entered the scene.
The United States had a series of two presidential administrations with polar opposite views on criminal justice.
The first, the Obama administration from 2009 to 2016, focused on reform.
It was followed by the Trump administration from 2017 to 2020, which concentrated on prosecution.
And this series of two administrations took forensic science on a wild ride.
Under Obama, a commission called the President's Council of Advisors advisors on science and technology researched and wrote a 2016 report called Forensic Science in Criminal Courts, Ensuring Validity of Feature Comparison Methods, which found many techniques, such as bite mark and shoe print analysis, so devoid of foundational validity that they should be prohibited.
But what was to be done with these findings would be left to the Trump administration.
What that turned out to be was that the Obama Commission's charter was allowed to immediately expire, the new Attorney General outright dismissed their report and said it would not be adopted, and Trump's own forensic science review was announced, which consists of one former prosecutor hired to write one release stating the Obama report was in error and should be ignored.
As of this writing, that's where we remain today.
But let's change gears now and have a closer look at some of these specific forensic disciplines.
All we've got time for on this short format show is a quick look, so by no means are these summaries intended to be comprehensive.
Why You Should Always Be Skeptical 00:10:52
As always, if you want the full details, come to the transcript page for this episode and look up the references for the full commission reports on these and more.
Hey everyone, I want to remind you about a truly unique and once-in-a-lifetime adventure.
Join me and Mediterranean archaeologist Dr. Flint Dibble for a skeptoid sailing adventure through the Mediterranean Sea aboard the SV Royal Clipper, the world's largest full-rigged sailing ship.
This is also the only opportunity you'll have to hear Flint and I talk about our experiences when we both went on Joe Rogan to represent the causes of science and reality against whatever it is that you get when you're thrown into that lion pit.
We set sail from Málaga, Spain on April 18th, 2026 and finished the adventure in Nice, France on April 25th.
You'll enjoy a fascinating skeptical mini-conference at sea.
You'll visit amazing ports along the Spanish and French coasts and Flint will be our exclusive onboard expert sharing the real archaeology and history about every stop.
We've got special side quests and extra skeptical content planned at each port.
This is a true sailing ship.
You can climb the rat lines to the crow's nest, handle the sails.
You can even take the helm and steer.
This is a real bucket list adventure you don't want to miss.
But cabins are selling fast and this ship does always sell out.
Act now or you'll miss this once-in-a-lifetime opportunity.
Get the full details and book your cabin at skeptoid.com slash adventures.
Hope to see you on board.
That's skeptoid.com slash adventures.
DNA It's best to open by pointing out that DNA evidence by itself is not some holy panacea.
DNA evidence can be used and it can be misused and often is.
One type of DNA analysis has been found to lack foundational validity and this is called combined probability of inclusion methods on complex mixtures.
This is where a complex mixture potentially containing genetic material from multiple individuals is given a probability that the suspect may be a contributor.
It's a sort of lightweight statistical analysis that depends on the examiner to make subjective choices which drive the final result.
And subsequent reanalysis have found the probabilities given in court to be way off in many cases, sometimes by factors of billions.
Ballistics.
One of the most common types of forensic evidence we've gotten used to is looking at the scratch marks on a bullet and supposedly proving that it was fired by this exact gun.
They take a bullet from a crime scene and compare it to one test fired from the gun under suspicion.
Well, it turns out that when examiners' results are tested against known results, their error rates are crazy high.
Various papers I read found error rates as low as 10% and as high as 66%.
In 2008, the National Research Council published a 344-page report called Ballistic Imaging, and one representative snippet from it says, The validity of the fundamental assumptions of uniqueness and reproducibility of firearms-related tool marks has not yet been fully demonstrated.
Bullet-led.
In some cases where ballistics hasn't been possible, such as when no gun was recovered, crime labs would do a bullet-lead examination to show that, for example, the bullet from the victim's body came from the same batch of bullets found in the suspect's garage.
The lab would find that the makeup of the lead was a match, and the suspect would be convicted.
However, what they never mentioned is that there are probably countless batches of bullets with that exact same chemical makeup.
This one has been a rare victory, with the FBI laboratory announcing in 2005 that they would no longer do this type of examination and present it as evidence.
Bite marks.
Bite mark comparison, called forensic odontology, is particularly unreliable.
The expert looks at bite marks on a victim and then compares them to the teeth of a suspect.
The fundamental problem is that while it can reasonably be used to exclude certain suspects whose teeth are totally different from the bite mark, it is typically used to pin the blame on a suspect.
There is no basis at all for this, as there is no evidence that dentition is unique, no evidence that dentition transfers a unique pattern onto skin which will retain it free of distortion, and strong evidence that different experts make different findings from the same data.
No large-scale study has ever been performed to see whether bite marks are unique, or to establish what percentage of the population could have produced any given bite mark.
Hair analysis.
Microscopic hair analysis, distinct from DNA analysis, which can't be done when you don't have the root, is predicated on the notion that hair from different people looks visually distinct under the microscope.
The Department of Justice has supported its use largely on the strength of a series of studies from the 1970s, which found a false positive rate of less than 1 in 40,000.
They've always ignored that the studies were savaged in the scientific journals for being deeply flawed methodologically, even making basic arithmetic errors in their conclusions.
When the FBI did a large study in 2002 to assess their own accuracy, they found an 11% false positive rate.
And the Department of Justice's response was to simply ignore this as well.
Fingerprinting.
Among the oldest and most famous forensic techniques, latent fingerprint analysis is considered to be foundationally valid, but has weaknesses jurors are unaware of.
First, the number of properly designed studies of the validity of fingerprinting is, guess how many?
Exactly two.
One found false positives occur in one in 306 matches, the other found one in 18.
The methodology used is sound, but the researchers found its practice is plagued with confirmation bias, examiners working backward from a suspected match, and matching more patterns that are actually present in the latent print.
Now to reiterate, that was just one very brief paragraph on six forensic disciplines.
There are many more, and the commissions have similar things to say on each of them.
My favorite summary of the state of forensic science is this lone sentence from the introduction to the 2009 National Academies report.
With the exception of nuclear DNA analysis, no forensic method has been rigorously shown to have the capacity to consistently and with a high degree of certainty demonstrate a connection between evidence and a specific individual or source.
That is an extremely bold and weighty statement.
And as such, it really underscores the value of having a basic science literacy and exposure to the value of skeptical analysis of pop culture beliefs like this one.
The average person on the street probably doesn't have the slightest inkling that there's a thing wrong with all that fancy forensic science they see on their favorite TV dramas.
And if that person gets onto a jury, he'll be the easiest rube in the room for some prosecutor who's hanging his case on carpet fibers and shoe prints, but who won't happen to mention that half the town could be convicted on that same basis.
Forensic science is yet one more example of why, whenever you run into something that everyone just assumes is true, but that has even the slightest of red flags, you should always be skeptical.
But you know who never makes any false assumptions?
Skeptoid's premium supporters like Mark Kessler, Peter Cootey from Wollongong, Australia, Gail Knapp, and Abra Sonnenstein.
Remember, Skeptoid is a 501c3 educational nonprofit.
We depend on monthly micropayments from our premium members to keep this content available to all the people out there who need it.
Join us in this mission.
Just come to skeptoid.com and click Go Premium.
And while you're at it, pick up an autographed Skeptoid book or a t-shirt or other cool stuff in our online store.
We've got all the styles, sizes, and colors at skeptoid.com slash store.
You're listening to Skeptoid, a listener-supported program.
I'm Brian Dunning from Skeptoid.com.
Hello, everyone.
This is Adrienne Hill from Skookum Studios in Calgary, Canada, the land of maple syrup and moose.
And I'm here to ask you to consider becoming a premium member of Skeptoid for as little as $5 per month.
And that's only the cost of a couple of Tim Horton's double doubles.
And that's Canadian for coffee with double cream and sugar.
Why support Skeptoid?
If you are like me and don't like ads, but like extended versions of each episode, premium is for you.
If you want to support a worthwhile nonprofit that combats pseudoscience, promotes critical thinking, and provides free access to teachers to use the podcast in the classroom via the Teacher's Toolkit, then sign up today.
Remember that skepticism is the best medicine.
Next to giggling, of course.
Until next time, this is Adrienne Hill.
From P R X
Export Selection