All Episodes
Previous Next
Plain Text
Nov. 1, 2016 - Skeptoid
17:43
Skeptoid #543: White Hat Journal Hoaxes

Sometimes the best way to scrutinize an open access journal is to hoax them. Learn about your ad choices: dovetail.prx.org/ad-choices

Transcriber: nvidia/parakeet-tdt-0.6b-v2, sat-12l-sm, and large-v3-turbo
|

Time Text
White Hat Hoaxing and Predatory Journals 00:06:42
Imagine writing a scientific paper that's deliberately full of nothing but nonsense.
Or even imagine programming a computer to automatically generate such a paper, more nonsensical than any human could manage.
And then, once you've done that, imagine sending this paper to actual scientific journals to try and get it published.
Is there ever a case where this might be the right thing to do?
We're going to find out right now on Skeptoid.
Hi, I'm Alex Goldman.
You may know me as the host of Reply All, but I'm done with that.
I'm doing something else now.
I've started a new podcast called Hyperfixed.
On every episode of HyperFixed, listeners write in with their problems and I try to solve them.
Some massive and life-altering, and some so minuscule it'll boggle your mind.
No matter the problem, no matter the size, I'm here for you.
That's HyperFixed, the new podcast from Radiotopia.
Find it wherever you listen to podcasts or at hyperfixedpod.com.
You're listening to Skeptoid.
I'm Brian Dunning from Skeptoid.com.
White Hat Journal Hoaxes.
The idea of having scientific papers published in respected journals is a good one.
Not only does it provide published access to the research, but association with a respected journal tells readers that this research paper is high-quality science.
Accordingly, the publishing industry has flourished as researchers have scrambled to have their papers published, and editors have kept the publication bar high to maintain their all-important reputation.
And quite predictably, this thirst for publication has been viewed as a business opportunity to some.
Predatory journals have appeared, charging authors money and performing little serious review of their articles.
Readers aren't necessarily able to know the quality of a journal and its contents, so the problem of predatory journals has become a serious one for academia at large.
One response conjured up by some authors to call attention to this is white hat hoaxing.
Submitting a deliberately bad paper to journals, hoping to get it approved and published.
And when it does, the excrement can hit the rotor.
The reasoning goes that if a journal can be publicly exposed as having published terrible work, it'll serve both to steer researchers away from using that publication as a resource and, hopefully, motivate that journal to improve its rigor.
The classic instance of a white hat hoax was in 1996.
Physicist Alan Sokol had grown weary of the content of a progressive humanities journal published by Duke University called Social Text.
He figured that they would publish pretty much anything so long as it sounded sufficiently anti-capitalist and was pumped up with stuffy academic jargon, the so-called intellectual elitism of the progressive left, or as he put it himself, flattered the editor's ideological preconceptions.
It turns out he was right.
Sokol wrote a nonsense article titled Transgressing the Boundaries Towards a Transformative Hermeneutics of Quantum Gravity, in which he argued that quantum gravity is politically liberal.
It was a coup of postmodernism, a dismissal of all science and reality as mere social constructs, and an embrace of intellectual idealism as a better explanation for anything than what mere scientific theory could ever hope to achieve.
When Sokol revealed the hoax following its publication, it was quite embarrassing for the social text editors.
They hemmed and hawed and condescendingly claimed that they knew the article was terrible, but published it as a gesture of kindness to an author seeking their approval.
Even their excuse showed that their publication criteria was anything but what it should have been.
Inspired by his success, Sokol co-authored a book titled Fashionable Nonsense: Postmodern Intellectuals' Abuse of Science.
Conversely, the editors of Social Text were awarded the 1996 Ig Nobel Prize in Literature for, quote, eagerly publishing research that they could not understand, that the authors said was meaningless, and which claimed that reality does not exist.
In a later explanation of why he perpetrated the hoax, Sokol wrote, My goal isn't to defend science from the barbarian hordes of literary criticism, we'll survive just fine, thank you, but to defend the left from a trendy segment of itself.
There are hundreds of important political and economic issues surrounding science and technology.
Sociology of science at its best has done much to clarify these issues.
But sloppy sociology, like sloppy science, is useless or even counterproductive.
And thus did white hat hoaxing become a real and effective tool to improve the quality of our scientific journals.
Inspired by this, a team of three Serbian academics, dismayed with the declining quality of Serbian journals, wrote a brilliantly nonsensical paper in 2013 titled, Evaluation of Transformative Hermeneutics Heuristics for Processing Random Data, and had it actually published in the journal Metallurgia International, a publisher with a poor reputation,
but that had been in business actually publishing monthly issues full of real papers for a long time.
The authors posed for their photos, published in the article, wearing goofy wigs and fake mustaches.
In their references, they cited porn star Ron Jeremy, king of pop Michael Jackson, fictional character Borat, someone named A.S. Hole, and of course, Alan Sokol, just in case anyone was paying attention, and in case their use of the word hermeneutics in their title wasn't an obvious enough homage.
Although Sokol and the Serbians caught their editors sleeping on the job, most white hat hoaxing is directed at journals that are not merely lazy, but actually predatory.
Spreading Good Ideas Through Skeptoid Files 00:02:16
Open access journals are those that offer their content free online.
Since they don't charge for subscriptions, they usually have to charge the authors to publish their work.
Good open access journals still employ top standards and have thorough peer review, but there's now a growing segment of predatory open access journals, or POA, who aren't really interested in anything but taking money from researchers eager to publish.
Their quality can be shockingly abysmal.
In a world that can feel overwhelming, spreading thoughtful, evidence-based content is one of the best ways to make a positive impact.
Ask your local public radio station to air the Skeptoid Files, a 30-minute radio-friendly version of Skeptoid that pairs two related episodes promoting real science, true history, and critical thinking.
And in these challenging times for public media, we're offering these broadcasts for free to radio stations, available on the PRX Exchange or directly from Skeptoid Media.
It's an easy ask.
Just send a quick message to your station's programming director.
By helping to bring the skeptoid files to the airwaves, you'll help promote the essential skills we all need to tell fact from fiction.
Just go to your local station's website, find the programming director's email address, or just their general email address.
You can even use the telephone.
I know that might sound crazy.
It's an old legacy device that allows real-time voice communication.
I know that's weird, but hey, it's an option.
The world can feel chaotic, but you're not powerless.
When you promote critical thinking, you can help your community tell fact from fiction.
And that's how we shape a better future.
In uncertain times, spreading good ideas can make you feel helpful, not helpless.
Let's stand up for reason, truth, and understanding.
Together, get them to air the Skeptoid files from Skeptoid Media, available on the PRX Exchange, and they'll know what that is.
No discussion of this subject can omit the most famous case.
The Beale Ciphers Peer Review Scandal 00:06:44
Back in 2005, two guys at NYU, David Maziers and Eddie Kohler, grew tired of receiving spam conference invitations, so they responded with an academic paper that consisted solely of the phrase, get me off your effing mailing list, repeated 772 times.
They didn't use the word effing, they used something more colorful.
It was nicely formatted and referenced, and even included a flowchart and a graph.
Their paper was hanging around, sort of a fun novelty, until 2014, when Peter Vamplu at Federation University, Australia, thought he'd submit it to a predatory open access journal to test their review process.
It was the International Journal of Advanced Computer Technology, located in India.
Almost immediately, he received an email advising him that the Maziers-Kohler paper had been accepted and giving him all the banking information he'd need to transfer $150 to a Mr. Tej Pal Singh at the State Bank of India in Delhi.
The paper had even been peer-reviewed, apparently, with scores of good, very good, or excellent in all nine categories.
And the following comments from the reviewer, who is sadly not named.
A. Please use latest references in order to increase your paper quality.
B. Introduction part is precisely explained.
C. Kindly prepare your camera-ready version of the paper as per IJACT paper format.
And D. Use high-resolution images in order to increase the appearance of paper.
Needless to say, Vamplu elected to keep his $150, but the paper and the responses from the alleged journal have become somewhat legendary.
One of the gurus of the POA phenomenon is Jeffrey Beale, an academic librarian at the University of Colorado.
He's best known for Beale's List, an online archive of hundreds of, quote, potential, possible, or probable predatory scholarly open access publishers.
Vamplu's International Journal of Advanced Computer Technology has since folded and disappeared.
Unsurprising since it appears to have been a one-man operation.
But all of the many on Beale's list are currently in business and taking money from anyone who will give it.
Some appear to be surprisingly sophisticated.
In one particularly sobering case, mechanical engineer Alex Martin tried to see if he could get a paper written by his seven-year-old son published.
It was a fun collection of basic facts about bats.
Martin selected a publisher from Beale's list called International Journal of Comprehensive Research and Biological Science and submitted the paper, reformatted to look scholarly.
After complying with a request for a minor revision, Martin received an email advising him that the article was accepted for publication in the forthcoming January 2015 issue of the journal, along with an invoice for $60.
In his write-up of this adventure, Martin said, There was no need to potentially tarnish the reputation of a seven-year-old by having him published in a non-reputable journal and elected not to pay.
But the editor was insistent, repeating his request for payment and including a galley proof of the article.
But, incredibly, Martin saw that the entire text of the article had been replaced.
And by searching the web, he found that the replacement text was copied verbatim in its entirety from two legitimately published articles about bats from two different authors from 2001 and 2002.
It didn't even stop there.
Martin looked up a number of other articles published in the journal and found that 100% of those he examined consisted entirely of content previously published in other journals.
It appeared that this journal's practice was to take checks from anyone and then publish copy and pasted text from legitimate sources.
So how easy is it to get past the review processes of open access journals?
In 2012, the top-rated journal Science engaged Harvard biologist John Bohannon to perform a sting operation.
He wrote a research paper that appeared good enough to a layperson, but contained gross errors that any competent biologist would readily spot.
Errors clearly bad enough that it rendered the paper unpublishable.
He made up fictitious African authors and research institutions, as African scientists are often under the radar and it wouldn't raise suspicion if someone failed to verify them online.
Bohannon then went to the Directory of Open Access Journals, the DOAJ, which lists thousands of credible open access journals.
He also went to Beale's list.
In the end, he filtered his list down to 304 pay-to-publish journals.
He spent months carefully submitting the paper, randomizing everything he could.
When reviewers requested changes, he made them.
He followed each submission through until it was either rejected or accepted.
The results were ugly.
84% of Bohannon's submissions went all the way through to either acceptance or rejection.
Of those, 60% gave no evidence of having been peer-reviewed.
Of those publishers that did legitimately peer-review the paper, a paper that should not have survived any peer-review process, 70% accepted it.
Only 36 of those 304 submissions came back with comments recognizing the grave scientific errors.
And of those 36, 16 were accepted anyway.
Not surprisingly, 82% of those publishers that came from Beale's list were accepted.
But it is surprising that 45% of those from DOAJ were accepted also.
Those are supposed to be the legitimate open access journals.
Invoices for publication, none of which were ever paid, rained upon Bohannon, many charging thousands of dollars.
About a third of the invoices were from India, the evident hotbed of POA activity, but it's fair to note that there were also rejections from Indian publishers.
Supporting Science With Double Doubles 00:01:55
Obviously, there are still plenty of Martin's bats in the open access attic.
And as long as there are, I'm going to continue to stand by white hat hoaxing as a valuable force in the academic publishing industry.
The more papers get rejected for the right reasons, the stronger our scientific literature becomes as a whole.
The reverse is also true.
Our literature becomes weaker when poor papers are not rejected.
So long live the hoaxers and the genius of their nonsense.
You're listening to Skeptoid, a listener-supported program.
I'm Brian Dunning from Skeptoid.com.
Hello, everyone.
This is Adrian Hill from Skookum Studios in Calgary, Canada, the land of maple syrup and mousse.
And I'm here to ask you to consider becoming a premium member of Skeptoid for as little as five US dollars per month.
And that's only the cost of a couple of Tim Horton's double doubles.
And that's Canadian for coffee with double cream and sugar.
Why support Skeptoid?
If you are like me and don't like ads, but like extended versions of each episode, Premium is for you.
If you want to support a worthwhile non-profit that combats pseudoscience, promotes critical thinking, and provides free access to teachers to use the podcast in the classroom via the Teacher's Toolkit, then sign up today.
Remember that skepticism is the best medicine.
Next to giggling, of course.
Until next time, this is Adrienne Hill.
From PRX
Export Selection