All Episodes
Previous Next
Plain Text
Feb. 14, 2012 - Skeptoid
16:21
Skeptoid #297: A Magical Journey through the Land of Reasoning Errors

Four common types of analytical errors in reasoning that we all need to beware of. Learn about your ad choices: dovetail.prx.org/ad-choices

Transcriber: nvidia/parakeet-tdt-0.6b-v2, sat-12l-sm, and large-v3-turbo
|

Time Text
Early Years of Logical Fallacies 00:01:46
In the early years of the Skeptoid podcast, we covered all the basic logical fallacies and have done some follow-ups here and there, adding in a few new ones.
The more we know about the possible pitfalls in logical reasoning, the less likely we are to fall into them.
So today we've got for you a roundup of a related stumbling block, the four basic reasoning errors.
That's right now on Skeptoid.
A quick reminder for everyone, you're listening to Skeptoid, revealing the true science and true history behind urban legends every week since 2006.
With over a thousand episodes, we're celebrating 20 years of keeping it focused and keeping it brief.
And we couldn't have done it without your curiosity leading the way.
And now we're even offering a little bit more.
If you become a premium member, supporting the show with a monthly micropayment of as little as $5, you get more Skeptoid.
The premium version of the show is not only ad-free, it has extended content.
These episodes are a few minutes longer.
We get rid of the ads and replace them with more Skeptoid.
The Extended Premium Show available now.
Come to Skeptoid.com and click Go Premium.
You're listening to Skeptoid.
I'm Brian Dunning from Skeptoid.com.
A magical journey through the land of reasoning errors.
Today we're going to cover a bit of new ground in the basics of critical thinking and critical reasoning.
Types of Reasoning Errors 00:14:13
There are several defined types of common analytical errors to which we're all prone, some perhaps more so than others.
Reasoning errors can be made accidentally, and some can even be made deliberately as a way to influence the acceptance of ideas.
We're going to take a close look at the type 1 false positive error, the type 2 false negative error, the type 3 error of answering the wrong question, and finally, the dreaded type 4 error of asking the wrong question.
By way of example, we'll apply these errors to three hypothetical situations, all of which should be familiar to fans of scientific skepticism.
One, from the realm of the paranormal, a house is reported to be haunted.
The null hypothesis is that there is no ghost until we find evidence that there is.
Two, the conspiracy theory that the government is building prison camps in which to orderly dispose of millions of law-abiding citizens.
The null hypothesis is that there are no such camps until we find evidence of them.
And three, from alternative medicine, the claim that vitamins can cure cancer.
The null hypothesis is that they don't unless it can be proven through controlled testing.
So let's begin with...
Type 1 error, false positive.
A false positive is failing to believe the truth, or more formally, the rejection of a true null hypothesis.
It turns out there's nothing there, but you conclude that there is.
In cases where the null hypothesis does turn out to be true, a type 1 error incorrectly rejects it in favor of a conclusion that the new claim is true.
A type 1 error occurs only when the conclusion that's made is faulty, based on either bad evidence, misinterpreted evidence, an error in analysis, or any number of factors.
In the haunted house, type 1 errors are those that occur when the house is not, in fact, haunted, but the investigators erroneously find that it is.
They may record an unexplained sound and wrongly consider that to be proof of a ghost, or they may collect eyewitness anecdotes and wrongly consider them to be evidence, or they may have a strange feeling and wrongly reject all other possible causes for it.
The conspiracy theorist commits a type 1 error when the government is not, in fact, building prison camps to exterminate citizens, but he comes across something that makes him reject that null hypothesis and conclude that it is happening after all.
Perhaps he sees unmarked cars parked outside a fenced lot that has no other apparent purpose and wrongly considers that to be unambiguous proof.
Or perhaps he watches enough YouTube videos and decides that so many other conspiracy theorists can't all be wrong.
Perhaps he simply hates the government, so he automatically accepts any suggestion of their evil doing.
Finally, the alternative medicine hopeful commits a type 1 error when he concludes that vitamins successfully treat a cancer that they actually don't.
Perhaps he hears enough anecdotes or testimonials.
Perhaps he's mistrustful of medical science and erroneously concludes that alternative medicine must therefore work, or whatever his thought process is, but an honest conclusion that the null hypothesis has been proven false is a classic type 1 error.
Type 2 error, false negative.
Cynics are those who are most often guilty of the type 2 error, the acceptance of the null hypothesis when it turns out actually to be false.
It turns out that something is there, but you conclude that there isn't.
If you actually do have psychic powers, but I'm satisfied that you do not, I commit a type 2 error.
The villagers of the boy who cried wolf commit a type 2 error when they ignore his warning, thinking it false, and lose their sheep to the wolf.
The proto-human who hears a rustling in the grass and assumes it's just the wind commits a type 2 error when the panther springs out and eats him.
Perhaps somewhere there is a house that actually is haunted and maybe the TV ghost hunters find it.
If I laugh at their silly program and dismiss the ghost, I commit a type 2 error.
If it were to transpire that the government actually is implementing plans to exterminate millions of citizens in prison camps, then everyone who has not been particularly concerned about this, myself included, has made a type 2 error.
The invalid dismissal of vitamin mega-dosing would also be a type 2 error if it turned out to indeed cure cancer, or whatever the hypothesis was.
Type 1 and 2 errors are not limited to whether we believe in some pseudoscience.
They're even more applicable in daily life, in business decisions and research.
If I have a bunch of skeptoid t-shirts printed to sell at a conference, I make a type 1 error by assuming that people are going to buy and it turns out that nobody does.
The salesman makes a type 2 error when he decides that no customers are likely to buy today so he goes home early, when in fact it turns out that one guy had his checkbook in hand.
Both type 1 and 2 errors can be subtle and complex, but in practice the type 1 error can be thought of as excess idealism, accepting too many new ideas, and the type 2 error as excess cynicism, rejecting too many new ideas.
In a world that can feel overwhelming, spreading thoughtful, evidence-based content is one of the best ways to make a positive impact.
Ask your local public radio station to air the Skeptoid Files, a 30-minute radio-friendly version of Skeptoid that pairs two related episodes promoting real science, true history, and critical thinking.
And in these challenging times for public media, we're offering these broadcasts for free to radio stations, available on the PRX Exchange or directly from Skeptoid Media.
It's an easy ask.
Just send a quick message to your station's programming director.
By helping to bring the Skeptoid files to the airwaves, you'll help promote the essential skills we all need to tell fact from fiction.
Just go to your local station's website, find the programming director's email address, or just their general email address.
You can even use the telephone.
I know that might sound crazy.
It's an old legacy device that allows real-time voice communication.
I know that's weird, but hey, it's an option.
The world can feel chaotic, but you're not powerless.
When you promote critical thinking, you can help your community tell fact from fiction.
And that's how we shape a better future.
In uncertain times, spreading good ideas can make you feel helpful, not helpless.
Let's stand up for reason, truth, and understanding.
Together, get them to air the Skeptoid files from Skeptoid Media, available on the PRX Exchange, and they'll know what that is.
Before talking about type 3 and 4 errors, it should be noted that these are not universally accepted.
Types 1 and 2 have been standard for nearly a century, but various people have extended the series in various directions since then.
So there's no real convention for what types 3 and 4 are.
However, the definitions I'm going to give are probably the most common, and they work very well for the purpose of skeptical analysis.
Type 3 error, answering the wrong question.
Types 3 and 4 are a little more complicated, but they're just as common and just as important to understand.
A type 3 error is when you answer the wrong question, and how this usually comes around is when you base some assumption upon a faulty or unproven premise, and so you jump one step ahead to solve a problem that isn't yet the question at hand.
The ghost hunters in the haunted house make a type 3 error when they start with the assumption that a ghost makes a cold spot in the room.
So they walk around the haunted house with all sorts of fancy thermometers and collect detailed temperature readings throughout the building.
This is great.
They've done fine work and documented it all very nicely, and they correctly reported temperatures.
However, it is a type 3 error because the question of temperatures has not yet been shown to be relevant since it has never been established that ghosts affect temperatures.
The conspiracy theorists commit a type 3 error when they publish a detailed list of all the locations they've identified as government prison camps.
The question is not yet, where are these camps?
Because they skipped over convincingly answering the precedent question of do such camps exist at all.
You can produce lists all day long, but until you first prove that each item on the list is actually what you claim it is, the list is of no value.
The vitamin salesman commits a type 3 error every time he answers a customer's question about what vitamin is best to take to treat or prevent cancer.
He'll no doubt give some such answer and recommend a particular supplement and perhaps recommend a dosage.
This is a type 3 error because he's ignoring and skipping the precedent question, which is whether the vitamin in question will treat or prevent the particular cancer in question at all.
Type 4 error, asking the wrong question.
While the type 3 error is usually committed innocently and with good intentions, the type 4 error, asking the wrong question, often suggests a deliberate deception.
By selecting the wrong question to investigate, it's possible to have greater control over the results.
Selecting the wrong question is a great way of diverting attention away from the right question.
The producers of ghost hunting TV shows know that they need to produce a program that yields positive results.
They also know that they're not going to happen to run into any ghosts or catch anything unexpected on camera.
So instead, they frame their program around asking the wrong questions.
Can we get interesting readings on our electrical and temperature meters?
By structuring their show around the wrong questions, they commit a deliberate type 4 error in order to produce the desired answers.
Conspiracy theorists of all flavors love the type 4 error, as it is one of the most effective tools to build arguments in support of non-existent phenomena.
If the conspiracy theorist wants to convince us that the government is building prison camps to enslave American citizens, it's not necessary to actually ask that question.
Instead, ask a whole assortment of related questions that are guaranteed to have positive answers.
Are there examples of government corruption?
Has the government imprisoned people in the past?
Are there laws that permit the government broader powers during times of emergencies?
Are there plots of land for which there is no obvious purpose?
These questions are all great type 4 errors for the conspiracy theorist.
Similarly, alternative medicine proponents can ask type 4 error questions to suggest that their central claims, which are unevidenced, are actually true.
Are there examples of corruption in big pharma?
Do any natural compounds have therapeutic value?
Do scientists rely on grant money?
Is medical science big business?
Again, these questions are all easily answered positively and appear to justify the use of vitamins to treat cancer, when in fact, none of them have any direct relevance to that.
And so, there we have it.
Four types of reasoning errors.
Four cases you've heard a thousand times and will hear a thousand more times.
Listen to a few sales pitches, watch a few documentaries on the pseudoscience TV channels, and see if you can spot them.
Chances are, you will.
And if you can develop enough familiarity with them to spot them when you hear them, you're a leg up on avoiding making these same errors yourself.
We all do it, and the better we understand the errors, the better prepared we are to minimize our own such failings.
Want more?
If you use Twitter, follow me at Brian Dunning to get the daily Skeptoid and much more.
You're listening to Skeptoid.
I'm Brian Dunning from Skeptoid.com.
Hello everyone, this is Adrian Hill from Skookum Studios in Calgary, Canada, the land of maple syrup and moose.
And I'm here to ask you to consider becoming a premium member of Skeptoid for as little as $5 per month.
And that's only the cost of a couple of Tim Horton's double doubles.
And that's Canadian for coffee with double cream and sugar.
Why support Skeptoid?
If you are like me and don't like ads, but like extended versions of each episode, Premium is for you.
If you want to support a worthwhile non-profit that combats pseudoscience, promotes critical thinking, and provides free access to teachers to use the podcast in the classroom via the Teacher's Toolkit, then sign up today.
Skepticism Is Best Medicine 00:00:17
Remember that skepticism is the best medicine.
Next to giggling, of course.
Until next time, this is Adrienne Hill.
From PRX
Export Selection