All Episodes
Previous Next
Plain Text
April 6, 2007 - Skeptoid
16:56
Skeptoid #37: How to Spot Pseudoscience

This 15-point checklist will help you tell science from pseudoscience. Learn about your ad choices: dovetail.prx.org/ad-choices

Transcriber: nvidia/parakeet-tdt-0.6b-v2, sat-12l-sm, and large-v3-turbo
|

Time Text
The 15-Point Skeptic's Checklist 00:07:07
From all the experience of debunking urban legends and false claims, I've developed a sort of toolkit that I use to help me tell what's real from what's not.
And today, I've assembled that toolkit into a 15-point checklist for you.
Next time you hear a questionable claim, ask these 15 questions.
I guarantee you'll wind up with a solid verdict on the veracity of that claim.
And that's coming up right now on Skeptoid.
A quick reminder for everyone, you're listening to Skeptoid, revealing the true science and true history behind urban legends every week since 2006.
With over a thousand episodes, we're celebrating 20 years of keeping it focused and keeping it brief.
And we couldn't have done it without your curiosity leading the way.
And now we're even offering a little bit more.
If you become a premium member, supporting the show with a monthly micropayment of as little as $5, you get more Skeptoid.
The premium version of the show is not only ad-free, it has extended content.
These episodes are a few minutes longer.
We get rid of the ads and we'll replace them with more Skeptoid.
The Extended Premium Show available now.
Come to Skeptoid.com and click Go Premium.
You're listening to Skeptoid.
I'm Brian Dunning from Skeptoid.com.
How to Spot Pseudoscience.
Welcome to the show that separates fact from fiction, science from pseudoscience, real history from fake history, and helps us all make better life decisions by knowing what's real and what's not.
Carl Sagan was one of the first to bring to the mass public a toolkit for deciphering science from pseudoscience when he published Demon Haunted World in 1996.
His baloney detection kit was a set of guidelines to help lay people spot logical fallacies and other common flaws in popular pseudoscientific claims.
Many other great minds have continued in this tradition, and today we're flush with similar lists.
In an effort to increase the confusion, I've taken it upon myself to compile some of the best, mix them all up, pick and choose the best points, and distill them down into a single list.
In compiling this list, I owe the biggest debts to Dr. Stephen Barrett of Quackwatch.org, Dr. Tom Pearls of anti-agingquackery.com, Dr. Michael Shermer of the Skeptic Society, and of course, Dr. Carl Sagan.
All four of these guys have put in many years of hard work protecting the innocent public from harmful and untruthful scientific claims.
It's no accident that common themes run through all of their work.
I've created a 15-point checklist that I call How to Spot Pseudoscience.
When you hear any claim about a new product, a new discovery, or some paranormal ability, run it through these 15 questions and you'll get a pretty clear idea of whether or not it has any merit.
Number one, does the claim meet the qualifications of a theory?
Very few claims that aren't true actually qualify as theories.
Let's review the four main requirements that a theory must fulfill.
Number one, a theory must originate from and be well supported by experimental evidence.
Anecdotal or unsubstantiated reports don't qualify.
It must be supported by many strands of evidence and not just a single foundation.
You'll find that most pseudoscience is supported by only a single foundation.
Number two, a theory must be specific enough to be falsifiable by testing.
If it cannot be tested or refuted, it can't qualify as a theory.
And if something is truly testable, others must be able to repeat the tests and get the same results.
You'll find that this feature is truly rare among pseudosciences.
They'll generally claim some excuse or make up a reason why it can't be tested or repeated by others.
Number three, a theory must make specific testable predictions about things not yet observed.
Number four, a theory must allow for changes based on the discovery of new evidence.
It must be dynamic, tentative, and correctable.
You'll find that most pseudoscience does not allow for changes based on new discoveries.
Number two, is the claim said to be based on ancient knowledge?
This is a sure sign that the claim is not based on scientific evidence, and it's intended to fool you into thinking that because the ancient Chinese believed it, it must have merit.
In fact, many true theories are not very old at all, because they've replaced older theories as knowledge has increased.
Generally, the more recent the finding, the better scientific foundation it has.
Number three, was the claim first announced through mass media or through scientific channels?
Real discoveries go through an unbiased peer review process, which results in publication through scientific journals.
When a belief is first announced through the mass media, like Pons and Fleischmann's cold fusion experiments, or like this Storn-Orbo perpetual motion machine, there's generally a reason its proponents chose not to subject it to the scrutiny of peer review.
Number four, is the claim based on the existence of an unknown form of energy or other paranormal phenomenon?
Loose, meaningless usage of a scientific sounding word like energy is one of the most common red flags you'll see on popular pseudoscience.
Terms like energy fields, negative energy, qi, orgone, aura, psi, and transdimensional energy are utterly meaningless in any scientific context.
Approach with extreme caution.
Number five, do the claimants state that their claim is being suppressed by authorities?
This is usually a really frail excuse for why mainstream scientists don't take their claim seriously, why the product is not approved by the FDA, or why scientific journals won't publish their articles.
You'll often hear this in the form of a conspiracy of the medical establishment to suppress a quack cure because it's in the interest of the medical industry to keep you sick.
In fact, any doctor or pharmaceutical company that could develop a new cure would make a huge fortune.
They'd never suppress it.
The same goes for auto manufacturers worldwide who are said to be suppressing new efficient engine technologies.
Number six, does the claim sound far-fetched or too good to be true?
Join the Sea Conference Adventure 00:02:36
When something sounds too good to be true, it usually is.
Extraordinary claims require extraordinary evidence.
Does the claim truly fit in with what we know of the way the world works?
How often do claims that turn the world upside down really turn out to be true?
Approach such claims with extreme skepticism and demand evidence that's as extraordinary as the claim.
Number seven, is the claim supported by hokey marketing?
Be wary of marketing gimmicks and keep in mind that marketing gimmicks are by themselves completely worthless.
Examples of hokey marketing that should always raise a red flag are pictures of people wearing white lab coats, celebrity endorsements, anecdotes and testimonials from any source, and mentions of certifications, colleges, academies, and institutes.
Hey everyone, I want to remind you about a truly unique and once-in-a-lifetime adventure.
Join me and Mediterranean archaeologist Dr. Flint Dibble for a skeptoid sailing adventure through the Mediterranean Sea aboard the SV Royal Clipper, the world's largest full-rigged sailing ship.
This is also the only opportunity you'll have to hear Flint and I talk about our experiences when we both went on Joe Rogan to represent the causes of science and reality against whatever it is that you get when you're thrown into that lion pit.
We set sail from Málaga, Spain on April 18th, 2026 and finish the adventure in Nice, France on April 25th.
You'll enjoy a fascinating, skeptical mini-conference at sea.
You'll visit amazing ports along the Spanish and French coasts and Flint will be our exclusive onboard expert sharing the real archaeology and history about every stop.
We've got special side quests and extra skeptical content planned at each port.
This is a true sailing ship.
You can climb the rat lines to the crow's nest, handle the sails.
You can even take the helm and steer.
This is a real bucket list adventure you don't want to miss.
But cabins are selling fast and this ship does always sell out.
Act now or you'll miss this once-in-a-lifetime opportunity.
Get the full details and book your cabin at skeptoid.com slash adventures.
Hope to see you on board.
That's skeptoid.com slash adventures.
Applying Occam's Razor Tests 00:04:40
Does the claim pass the Occam's Razor test?
Is there a simpler, natural explanation for the claim that does not require any supernatural component?
Are results consistent with the placebo effect or the body's natural healing capacity?
Can a stage magician duplicate the psychic's feats?
The law of large numbers states that a one-in-a-million event usually happens to everyone about once a month.
And since Occam's Razor says that the simpler of two possible explanations is usually the right one, don't leap for a supernatural explanation just because you happened to dream about your grandmother on the night she died.
Number nine, does the claim come from a source dedicated to supporting it?
Science works by starting with a null hypothesis and searching for evidence.
Pseudoscience starts with a positive hypothesis and supports it with questionable research and anecdotal reasoning.
It's unlikely that an institution dedicated to the promotion of any given claim will present any type of evidence other than that which supports their claim, and its bias should be given serious consideration.
Number 10.
Are the claimants upfront about their testing?
Any good research will outline the testing that was done and will present all evidence that did not support the conclusion.
Be skeptical of any claims that do not detail testing methodology that was thorough and responsible, including external verification and duplication, or that do not provide evidence unsupportive of the conclusion.
Number 11.
How good is the quality of data supporting the claim?
Watch out when testing data might be susceptible to observational selection, which is the counting of hits and not the misses, like we see with television psychics.
Watch out when sample sizes are too small to have statistical significance, as with most clinical trials of homeopathy.
And especially watch out for hastily drawn causal relationships, the assumption that because the relief occurred after the remedy, the remedy must have caused the relief.
Number 12.
Do the claimants have legitimate credentials?
Be aware that there's a huge number of unaccredited institutions, which are often just bedroom offices, giving out degrees in just about anything.
Be aware that some institutions claiming to be accredited receive their accreditation from unrecognized accreditation bodies.
Finally, be aware that genuine accredited universities often have programs in unscientific fields, such as chiropractic, naturopathy, and acupuncture.
You must be vigilant.
To see just how vigilant, go to thunderwoodcollege.com and get your own PhD in the field of your choice in seconds for free.
That's thunderwoodcollege.com.
Number 13.
Do the claimants state that there's something wrong with the norm?
When real research is presented, it consists of the evidence that was discovered and the conclusion.
It does not go off on alarmist rants about how the food we eat is dangerous, how we're destroying the planet, how the government covers up its evils, or how you're going to hell if you accept evolution.
When a claim is presented as an alternative to the wrongs of the status quo, it's a sign that the claim is probably based on ideology or philosophy rather than science.
Number 14.
Is the claim said to be all natural?
As we've seen time and time again, by no definition can all natural mean that a product is safe or healthy.
Consider the examples of hemlock, mercury, lead, toadstools, box jellyfish neurotoxin, asbestos, not to mention a nearly infinite number of toxic bacteria and viruses, E. coli, salmonella, bubonic plague, smallpox, SARS-CoV-2.
In many cases, synthetic versions of natural compounds have been engineered to make them safer, more effective, and able to be produced in large quantities.
Number 15.
Does the claim have support that is political, ideological, or cultural?
Some claimants suggest that it's moral, ethical, or politically correct to accept their claims, to redirect your attention from the fact that they may not be scientifically sound.
Skepticism Is Best Medicine 00:02:29
In some cases, such as young earth creationism, proponents use the court system to force schools to teach their claims as fact.
Generally, when a theory is scientifically sound, even if it's brand new, it will eventually find its way into the educational curriculum.
Good science is done in the lab and in the field, not in the courts, not in protest marches, not in blogs, and not in church.
A political or cultural campaign to legalize or promote some product or claim is a major indicator that it's bogus.
There you have it.
With this checklist, anyone is well equipped to filter out the chaff from the wheat.
Questions like these are what should be taught in schools, encouraging young people to begin looking at all the crazy misinformation in our world with critical analysis.
The ability to tell fact from fiction is essential to our progress as a species as we search for the next great discoveries in medicine, space exploration, computing, power generation, and every other scientific field.
You're listening to Skeptoid, a listener-supported program.
I'm Brian Dunning from Skeptoid.com.
Hello, everyone.
This is Adrian Hill from Skookum Studios in Calgary, Canada, the land of maple syrup and mousse.
And I'm here to ask you to consider becoming a premium member of Skeptoid for as little as $5 per month.
And that's only the cost of a couple of Tim Horton's double doubles.
And that's Canadian for coffee with double cream and sugar.
Why support Skeptoid?
If you are like me and don't like ads, but like extended versions of each episode, Premium is for you.
If you want to support a worthwhile nonprofit that combats pseudoscience, promotes critical thinking, and provides free access to teachers to use the podcast in the classroom via the Teacher's Toolkit, then sign up today.
Remember that skepticism is the best medicine.
Next to giggling, of course.
Until next time, this is Adrian Hill.
From PRX.
Export Selection