All Episodes
Previous Next
Plain Text
Dec. 27, 2011 - Skeptoid
14:52
Skeptoid #290: Approaching a Subject Skeptically

My process for examining a new topic, to learn whether it's fact or fiction. Learn about your ad choices: dovetail.prx.org/ad-choices

Transcriber: nvidia/parakeet-tdt-0.6b-v2, sat-12l-sm, and large-v3-turbo
|

Time Text
Approaching Subjects Skeptically 00:05:27
I've covered a lot of urban legends and strange beliefs on the show over the years, and one question I get all the time is, how do I go about approaching each subject?
Not necessarily the nuts and bolts of the research, but approaching dubious subjects with the right mindset.
So you don't come in as a cynic, and you certainly don't want to come in overly credulous.
We're going to talk about that today on Skeptoid.
Hi, I'm Alex Goldman.
You may know me as the host of Reply All, but I'm done with that.
I'm doing something else.
No.
I've started a new podcast called Hyperfixed.
On every episode of HyperFixed, listeners write in with their problems and I try to solve them.
Some massive and life-altering, and some so minuscule it'll boggle your mind.
No matter the problem, no matter the size, I'm here for you.
That's HyperFixed, the new podcast from Radiotopia.
Find it wherever you listen to podcasts or at hyperfixedpod.com.
You're listening to Skeptoid.
I'm Brian Dunning from Skeptoid.com.
Approaching a subject skeptically.
One of the questions I get asked a lot is how I go about approaching a new subject.
When you hear about something new, what's the best way to think about it?
What's the best way to determine whether it's science or pseudoscience?
Well, I'm not sure that there is a best way, and I don't think there's one methodology that everyone can follow that's going to work in every circumstance.
But I'll try to give the best answer I can.
It's probably not the same answer you'll hear from others, but this is what works for me.
First of all, and perhaps most important, is that there's a separation between my daily life and working on Skeptoid.
I don't walk around demanding peer-reviewed scientific evidence for everything that I see.
I don't have a crazed, obsessive drive to know the validity of every new product for sale at the mall.
I'd never get through my day without a certain amount of tolerance for pseudoscience.
Fad products, marketing campaigns, green washing, and even straight-up fraudulent claims surround us all day, every day.
I accept that.
Trying to be a full-time challenger of pseudoscience would not only be hopelessly chaotic, it would also annoy everyone around me and rob me of the freedom to enjoy my day.
So I let virtually everything slide.
A co-worker is wearing a magic bracelet?
Great, good for him.
Neighbor talks about her great visit to the reflexologist?
Bully for her.
Overhear some people discussing what Nostradamus said about the 2012 apocalypse?
Whatever floats their boat.
But what if I'm out with friends and somebody asks me my thoughts on something?
This happens all the time.
You're on the spot.
You don't have access to research materials.
You don't have the time to look into it.
Now, oftentimes I've already done an episode about the subject in question, or something really similar.
It gives me a pretty good foundation.
Sometimes I haven't, and like most people, have to rely on a journeyman's knowledge of a subject area that's outside of my core competence.
This provides a pretty good overview of whether or not the new claim is in line with what's generally known about the subject.
Usually it's not, otherwise it wouldn't be on the news or wherever it was that my friends heard about it.
So there you are.
You're given something that raises your skeptical radar.
It's outside your core competence, and your friends just saw it on television or the internet.
Despite the fact that most people say they take TV or internet reports with a grain of salt, few actually do.
There's something deeply compelling about hearing a claim from an authoritative source.
We all have a voice in the back of our heads that wants the new claim to be true, and this desire gets confirmed by the belief that the story wouldn't have made it all the way to the TV news without having been pretty well substantiated.
What are you going to do?
The first thing I'd do is take out my phone and track down the original source of the story, using keywords from the report to search Google.
I'd want to know if it was reported in any journals, or if it skipped this process and went straight to the mass media.
This is the simplest and fastest way to see if a new claim or phenomenon has come from the world of legitimate research or if it comes from a crank, charlatan, or manufacturer operating outside of science.
You always have to remember that the mass media doesn't care.
They're interested in the sensationalism of the story, not in its validity.
That's it.
That's probably all I'm going to do when I'm out in the world and get a question that's worthy of looking into.
It's not a perfect process, but nine times out of ten, this will correctly tell you whether there's something there or whether it's just more noise from the media clamoring for eyeball share.
It's only when I take my seat in the skeptoid office that I assume the mantle of proper separator of fact and fiction.
This is when I take each week's topic and give it my honest best effort at a good skeptical treatment.
The best topics are those that are popularly misunderstood, but with facts behind them that, when properly understood, are way cooler than the popular version.
This isn't as hard as it might sound.
Nearly every popular myth has some history that puts its genesis into a fascinating new perspective.
Sailing Into Adventure 00:03:06
Sometimes finding this perspective takes me back in time to an out-of-print book or to a newspaper article a century old.
Tracking these down requires a lot of e-book purchases, Google Books downloads, newspaper archive searches, and occasionally even the coveted trip to a real library to find a real book.
Of course, even the relevant pages from the real book end up as electronic files on my computer, photographed with the iPhone and then OCR converted to searchable text.
Getting brand new information, like current research, is almost exactly the same process.
It's all available when you have the right accounts to access online research libraries.
But none of that compares to the few chances to actually go in person to a place where something strange is said to have happened.
To smell the dusty desert wind across Death Valley's racetrack playa.
To touch the cold granite of the Georgia Guidestones, and to photograph a Fata Morgana mirage, such as the ones responsible for so many legendary ghost lights.
Hey everyone, I want to remind you about a truly unique and once-in-a-lifetime adventure.
Join me and Mediterranean archaeologist Dr. Flint Dibble for a skeptoid sailing adventure through the Mediterranean Sea aboard the SV Royal Clipper, the world's largest full-rigged sailing ship.
This is also the only opportunity you'll have to hear Flint and I talk about our experiences when we both went on Joe Rogan to represent the causes of science and reality against whatever it is that you get when you're thrown into that lion pit.
We set sail from Malagas, Spain on April 18th, 2026 and finish the adventure in Nice, France on April 25th.
You'll enjoy a fascinating skeptical mini-conference at sea.
You'll visit amazing ports along the Spanish and French coasts and Flint will be our exclusive onboard expert sharing the real archaeology and history about every stop.
We've got special side quests and extra skeptical content planned at each port.
This is a true sailing ship.
You can climb the rat lines to the crow's nest, handle the sails.
You can even take the helm and steer.
This is a real bucket list adventure you don't want to miss.
But cabins are selling fast, and this ship does always sell out.
Act now, or you'll miss this once-in-a-lifetime opportunity.
Get the full details and book your cabin at skeptoid.com slash adventures.
Hope to see you on board.
That's skeptoid.com slash adventures.
I've been doing this show every week for five years now, and on the one hand, you might assume that I've developed a certain aptitude for smelling rats and have pretty good radar for science versus pseudoscience.
Well, that's true to a degree, but at the same time, I've learned that I can easily be surprised.
Validating Stories Thoroughly 00:06:15
I often learn that something that sounded pretty hokey is actually true, and something I took for granted turns out to be false.
So rather than having developed a super sense for fact and fiction, I've actually picked up a more acute awareness of my own ignorance.
Kind of the opposite of what one might hope for, but as we see so often, magically easy solutions to complex problems are a fool's gold.
The process is different every time, but it always starts with a quick survey of the most popular sources, followed by delving deeper into the roots.
If it's homeopathy, I want to know what led Samuel Hahnemann to his original conclusions.
If it's a conspiracy theory, I want to know who came up with it and what question they were trying to answer.
If it's a ghost story, I want to know who first wrote about it and what their relationship was to the hauntee.
It's critical to allow for the possibility that the story may or may not be as reported, and to follow up the leads in both directions.
Frequently, this requires some pretty detailed departure from the popularly known core of the story.
For example, say you find a reference to the mayor of an old town.
First, you find out if the town actually exists, where it was, whether it's still there, and find it on Google Earth to see if it makes sense within the context of the story.
Then, find out if the person listed as the mayor actually was the mayor.
Find out when he was born to see if the timing is right.
There are a myriad details you can drill down through to be as thorough as possible validating the story.
Sometimes there are an endless number of these leads, and with only a week between episodes, I often have to simply stop following them, thus making many episodes necessarily incomplete and open to error.
But when you have the time, how far do you go tracking these leads?
I've found that there's never a point of diminishing returns.
Every time I've made a discovery or connection that, to my knowledge, no other researcher has found, it's always in one of these fine tails of data.
The unturned stones are rarely in the middle of the road most traveled.
They're in the obscure newspaper article that never got syndicated.
They're in the out-of-print interview with the expert who was misquoted in the popular version of the story.
And more than anywhere else, they're in the actual published research that was omitted from the mass media reports because it did not support a sensational revisioning of the story.
Now, I don't mean to sound cynical about the mass media.
There are many, many excellent reporters and news bureaus who conscientiously produce exceptional material.
But I think you'll find that the better they are, the more likely they are to give you an honest assessment of the industry's overall goal, which is to be profitable.
The easiest way to do this, as practiced by a probable majority of editors, is to be sensational.
I don't think it's a cynical assessment, and it has certainly proven itself to me time and time again through my work validating mass media reports.
So take the road most traveled, as presented in Wikipedia, to get the lay of the land.
But to truly learn anything new, you must explore those obscure details that nobody else had the time for or that they overlooked.
Interestingly, I'd say that my process, though it's much more thorough, is probably no more accurate than the quick trick in the restaurant with a smartphone and Google.
The more information I collect, the more possibility for error.
The more obscure threads I follow, the more are likely to be unreliable.
And the more time I spend trying to be thorough on one part of the story, the less time I have for the other parts, an unfortunate exigency of producing a weekly show.
I'd say that errors of omission are my most common mistakes, followed by errors that I just didn't catch because of limited time.
And like every fallible biological entity, I also make errors by misinterpreting, misreading, and failing to see beyond my own personal biases.
You'll make these same errors in your own research.
The best defense against them is to acknowledge your blind spots, compensate for them, and honestly qualify remarks that you can't be sure of.
First, I try to be right more often than I'm wrong.
But second, I try to emphasize the process over the conclusions.
Being right nine times doesn't guarantee that you'll be right the tenth time.
But trying hard all ten times guarantees that you'll at least be as right as your process is capable of.
Follow Brian Dunning on Twitter for an extra daily dose of science and critical thinking.
You're listening to Skeptoid.
I'm Brian Dunning from skeptoid.com.
Hello everyone, this is Adrian Hill from Skookum Studios in Calgary, Canada, the land of maple syrup and mousse.
And I'm here to ask you to consider becoming a premium member of Skeptoid for as little as $5 per month.
And that's only the cost of a couple of Tim Horton's double doubles.
And that's Canadian for coffee with double cream and sugar.
Why support Skeptoid?
If you are like me and don't like ads, but like extended versions of each episode, premium is for you.
If you want to support a worthwhile non-profit that combats pseudoscience, promotes critical thinking, and provides free access to teachers to use the podcast in the classroom via the Teacher's Toolkit, then sign up today.
Remember that skepticism is the best medicine.
Next to giggling, of course.
Until next time, this is Adrienne Hill.
From PRX
Export Selection