DarkHorse Podcast with Tess Lawrie & Bret Weinstein
Tess Lawrie is an MD and PHD, external analyst for the World Health Organization, and an expert in analysis of medical evidence.The BIRD Group website: https://bird-group.org/The BIRD Group on Twitter: @BIRDGroupUK --- Find Bret Weinstein on Twitter: @BretWeinstein, and on Patreon. Please subscribe to this channel for more long form content like this, and subscribe to the clips channel @DarkHorse Podcast Clips for short clips of all our podcasts. DarkHors...
I have the distinct pleasure of sitting today with Dr. Tess Laurie, who is an MD and PhD.
She is also an external analyst for The Who and an expert in analysis, and as I understand it, Tess, also in the extrapolation of evidence to prescriptive medicine.
Is that a fair characterization?
It's called, well, it's a guideline methodologist, so I help to assess evidence and compile it in a way in what's called evidence-to-decision frameworks to help guideline panels make recommendations on treatments.
Evidence-to-decision.
That is a dry way of describing how to understand what we believe we know and extrapolate to treating patients so they get healthier.
Fair enough?
All right, well, welcome to Dark Horse.
I certainly appreciate you making the time.
Now, I should say, you and I were scheduled to have a conversation today, and yesterday an article emerged in The Guardian that has thrown Twitter into chaos.
People seem to not understand how to interpret evidence and to change that interpretation based on Concerns, so that gives us the opportunity to help them sort it out.
Now, if you would allow me, I am not a data scientist.
I am a biologist, but I've certainly done analyses.
When I saw this Guardian article, a couple of things occurred to me.
One, the article is bizarrely political, and in fact politicizes the entire question.
So the article describes reasons for concern about a randomized control trial involving the drug ivermectin, which those who have followed Dark Horse will recognize, people who have followed you,
Dr. Laurie will also recognize, and so the long and short of it is, concerns were raised about a paper that reports that ivermectin is a useful treatment and prophylactic for COVID, and this caused many people, including the Guardian itself, to imagine that something about the overall picture of the utility of this drug had radically changed.
Now, my thinking was that's not the case.
That, in fact, if one believes that the only kind of evidence that matters is a large-scale randomized controlled trial, then yes, the discovery that a given trial is compromised is a very serious matter.
On the other hand, that's not the way evidence works.
Your expertise covers what's called meta-analysis.
Meta-analysis is the art of taking numerous different types of studies and figuring out how to properly integrate the findings so that they compile to a clearer picture.
And, am I correct in saying that having done a meta-analysis, if one finds a given study is suspect, that one can simply ask the meta-analysis to recalculate with that study removed?
Yes, one would do what's called a sensitivity analysis to see whether removing the study makes a difference to the overall estimate of effect.
And so, which study is it that has been called into question?
Well, there's a study called by Professor Al-Ghazal and his team in Egypt that looked at both ivermectin for treatment and ivermectin for prevention.
In our meta-analysis, we evaluate all studies in the same way following a systematic approach, and we assess risk of bias for each study And risk of bias can be assessed as low risk, high risk or unclear risk where there is some uncertainty and we can't get to positioning it either as low or high.
And we include all the study data in the analysis.
And then if there is a question of studies being high risk of bias, we remove them and see if that changes the results.
So you included this study in your meta-analysis.
How did you rank it in terms of likelihood of bias?
We ranked it as unclear risk of bias.
Unclear.
So you had concerns, but they were not specific.
Yes, we had had correspondence with
Professor Al-Ghazal during the course of our systematic review process, and there were some questions he addressed adequately, and there were some that we were still uncertain about, and we couldn't kind of get a full clarification, and we, you know, it is possible that it could be a language difference, and we weren't happy to call it high risk of bias, and we weren't happy to call it low risk, so it landed as an unclear risk of bias study.
All right, so it seems to me that there are two things that we have to do right up top here.
One is to say neither you nor I know whether the problems that have been pointed out in this study are evidence of some deep failure, something like fraud, or not, and that it does not make sense to rush to judgment, especially in light of the fact that we have
A highly regarded professor and a team of people published on this, and we don't know what the source of the concerns ultimately is.
We also don't know how much of an opportunity they had to respond to these concerns before The Guardian went to press.
Is that fair?
Yes, I wouldn't like to prejudge.
Professor Alcazar was always very responsive to our queries when we were going through the review process.
And, you know, I think when Professor Emeritus of a university's reputation is at stake, I think he deserves a fair opportunity to respond to the allegations.
Good.
Alright, so on the one side, in terms of impugning people's reputations, we need to be extremely careful.
And on the other side, we also need to be cautious not to derive any conclusion from a study that we have reason to be concerned about.
What happens when you remove this study from the meta-analysis?
In fact, maybe you can even show us what happens, given that meta-analysis is a method, and you can simply ask the method to exclude a given study.
What happens when you do exclude the study?
Yeah, sure.
Well, we have done that today, and we did find that the overall effect still clearly favors ivermectin.
And this is the beauty of meta-analysis.
So I'll show you, actually, if you don't mind just giving me a minute.
Not at all.
Not a minute.
Hopefully it won't be as long as that.
There we go.
Can you see my screen?
I definitely can.
Okay, so this is a program called Review Manager and it's used, it's a Cochrane, it's a free program actually and anyone can actually load the data and check it themselves if they wanted to, but the studies are listed, study names are listed down the left and these are the data for ivermectin and the data for the control group and this analysis is for the outcome of death, so it's the most important
outcome with regard to COVID.
So if we remove Al-Ghazal, which is this one, and it's over there as well.
Sorry, I should have said before I do that, I should just say, so this is the effect estimate here.
It's 0.38, which corresponds to a 62% reduction in death.
This is with the, the suspect study included?
Included, yes.
So if we remove the data, We get a broader confidence interval and the reduction is a 49% reduction in death, but it's still clearly to the left in favor of ivermectin.
So it hasn't changed our conclusions.
The heterogeneity is this figure here.
It's come down a little bit.
And it's perhaps not as dramatically significant as it was before.
It's a broader confidence interval, but it still wouldn't change our conclusions, or I believe other meta-analyses.
So, broader confidence interval means that your estimate has to take on a broader range because some data has been eliminated.
So, data that would give you confidence in how effective ivermectin was has been removed from your meta-analysis, meaning that your confidence that you know how useful ivermectin is has dropped, but you still have a very clear signal that it is quite valuable in terms of preventing people from dying of COVID.
Yes, yeah, it's still a clear signal.
It's just it's the precision of the estimate that's unclear.
So if we were to do the study a thousand times, the Real estimate would lie somewhere in that range.
That's how it's interpreted.
Right, and so...
It could be as dramatic, looking at these numbers, it could be as dramatic as 73% reduction in deaths, or as little as 5% reduction in deaths, but it is a clear reduction.
And I should point out a couple things.
One, people do not often understand the distinction between precision and accuracy.
And what you're saying about the confidence interval is that the precision of your estimate has formally dropped.
But it doesn't mean that the effectiveness of the drug is necessarily less.
In fact, I would point out here that Well, one thing that is logically true is a meta-analysis will include a range of methodologies of administration of the drug.
So the best you can do is whatever the optimal way of delivering the drug is, and then other studies will deliver it in some way that is suboptimal.
And so my point is It should not be assumed that 49% is the amount of reduction in death.
That is across studies that will range from good to not so good in terms of how they applied the drug.
Is that fair?
Yes, absolutely, because some used very low doses of ivermectin, as now we know that higher doses work better.
And some were in very sick people, and we now know that the sooner ivermectin is employed, the better.
And also some of these studies compared people who were on many different treatments, especially with the more severely sick people.
And so there were a whole range of medicines that the control group was receiving as well that might have been effective.
I would say that the effect estimate that we're seeing in this meta-analysis in particular is underestimated.
It's likely to be underestimated because you can't do better than the optimal protocol, but you can definitely do worse.
And in fact, virtually every protocol will be worse than optimal.
So what we would discover if we actually applied the drug is how you best use it.
Okay.
I think one of the things that I'm seeing, so I find myself strangely under attack here because there is now a concern about a particular study that shows ivermectin to be useful.
I don't believe I've ever pointed to this study.
I've certainly pointed to your meta-analysis, which includes this study.
But there is another leap to judgment that I see being made by many people, where they love to throw the idea of GEIGO, which is a computer science concept.
Garbage in, garbage out.
It is an absolute misapplication of this concept.
In fact, I have pointed out that one of the troubling things about randomized controlled trials is that if they are badly designed, they give you a very false sense of the truth of a given protocol.
Whereas a meta-analysis, because it takes from many different studies that will have different flaws, tends to neutralize those flaws and allows you to see the signal in spite of the fact that no trial will be perfectly designed.
And so, those who are thinking in this computer science way, GEIGO, well, we've got a study that may be compromised.
That means that anything that utilized that study is wrong, but that is incorrect.
In fact, what we have, as you've just shown us, is the ability to exclude a study that you may have new reason to suspect is unreliable, and
So what I would argue, and I don't mean to impugn this study or its authors, I do not know whether or not the study in the end has serious flaws or is fatally compromised by fraud or any such thing, but it's not Geigo, it's a question of if there is some garbage somewhere in a meta-analysis, you take the garbage out and you recalculate the result.
So far, is that fair?
Yes, it is.
Yeah, it's not the only study, you know, where we had some concerns and, you know, there have been in other studies, for example, there was a study by López Medina in Colombia that was highly criticised for the way it was conducted and the biases that were inherent in the study design.
And yet, You know, when you're doing a systematic review, you just evaluate all the studies using the same criteria, the same evaluation for risk of bias.
And, you know, when you pull it all together, you then grade the certainty of the evidence using a particular tool called the Grade Working Group Tool.
And that takes into account then how certain you are of the evidence.
We downgraded the certainty of our evidence in this analysis for design limitations inherent in some of the studies, and that's how we ended up with moderate certainty evidence and not high certainty evidence.
Right, yes, I noticed that when I read your study, that moderate certainty is a term of art, and in fact I contacted you at the time and asked you about it, and if I understood your answer correctly, what it means is your expectation of how likely your estimate is to move based on new evidence.
Is that correct?
Yeah, that's correct.
All right, excellent.
Now, I would also point out I've been struggling to call people's attention to a conflation that exists in the discussion of ivermectin and its utility, and that conflation is between its effect as a treatment and its effect as a prophylactic, effectively as a preventive measure to keep people from contracting COVID.
Now, the study that has been called into question studied both of these things, and therefore their data is included on both the treatment side and the prophylactic side of your meta-analysis.
So, I have been... I don't know whether you'll agree with this.
I suspect you will, but we'll find out.
I believe that as terrific as it is to have a drug, and it's not just one, there are several drugs that are repurposed that seem to have strong effectiveness in treating COVID.
As wonderful as it is to have drugs that do that, the more important effect is the ability to prevent people from contracting COVID.
And the reason that that's so important is that it offers the possibility of driving the pathogen to extinction.
And the difference in the amount of harm that will come to humanity from SARS-CoV-2 if it continues to linger and circulate annually versus goes extinct, that is a huge difference.
In fact, you can calculate it as a finite amount of harm that will have come from this pathogen if it goes extinct.
It is an indefinitely large amount of harm if it continues to circulate and find new victims every year and mutate and all of the other things that it might do.
So, the question then is, what happens in your meta-analysis when you redo the part that tells us how effective it is as a prophylactic to remove the study that has now been called into question?
Okay, well I'm ready with that.
I'm going to show you my screen.
There are only three studies that contribute data to the Comparison of ivermectin versus a control for prophylaxis.
So this measures the outcome COVID infection.
And what we see across these studies is, I think this works out at about a 5% risk of infection versus a 30% or maybe it was 37% risk of infection if you calculate that.
And this estimate of 0.14, It corresponds with an 86% reduction in COVID infections with ivermectin.
So if we remove the Algazar data, it doesn't really change much.
We've got an 87% reduction in deaths and the confidence interval is pretty much the same as it was.
So Fascinating.
So let me try to unpack that a little bit for people who are not so experienced with the analysis of data.
You've got three studies that tell us how effective this is as a prophylactic, a preventive of the contraction of COVID.
When you eliminate the study that has been called into question, the estimate of how effective it is to prevent COVID, if I understood what you said correctly, goes up very slightly.
And the confidence is likely to have gone down, and it sounds like not very much.
Is that correct?
Yeah, that's correct.
Okay, so the reason, it will be counterintuitive to many people, that the estimate of how effective it is at preventing COVID actually goes up when you eliminate this, and the reason is because the study in question reported a slightly lower degree of effectiveness than the average of the other two studies, or not average, probably that's the wrong term, but the weighted average of the other two studies.
Yeah, it just alters it.
So, in essence, the punchline to this is, A, isn't meta-analysis beautiful?
That you can simply, you know, you wake up to a headline that says you've got a study that maybe you can't trust, and you can, with a couple of keystrokes, eliminate that study and see how much it should change your understanding of the effect of the medicine in question.
That's the first punchline.
And the second punchline is with respect to what at least I am arguing is the more important use of this medicine as a preventive measure.
Very slightly, and probably not meaningfully, increases the estimate with which we would imagine, the estimate of how effective it is, and slightly decreases our confidence in that number.
It doesn't decrease the confidence that there is an effect, but it broadens the range over which that effect might fall.
Yeah, I think the biggest error actually in the ivermectin story and evidence is the failure to take into account the other enormous body of evidence that's not randomized controlled trial or systematic review and meta-analysis.
Just on the point of the latter, I've never seen an intervention with so many reviews conducted on it in the space of a year.
I think we are looking at 11 or 12 now.
looking at 11 or 12 now.
And also so many studies published and ongoing.
So I could actually share, we've been working on a new model actually just to describe this because randomized controlled trials and systematic reviews represent just a teeny bit of the amount of evidence on ivermectin.
Yes, so I agree, and I think there is something strange going on with respect to what seems to be an obsession with randomized controlled trials in this case, when in fact we have multiple different kinds of evidence which ought to lend a great deal of confidence.
A randomized controlled trial can be well designed, in which case it's excellent at amplifying a weak signal, but it can also be badly designed.
If it has systematic error, it will give too much confidence of a wrong A wrong belief.
So it is far better to have multiple kinds of evidence.
So why don't you talk us through that?
Well, randomized controlled trials are very good at showing efficacy, but they're really not good at showing safety.
And so whilst we have them, because randomized trials are usually designed around efficacy endpoints, they're not designed around safety endpoints.
And especially if you've got rare side effects, But they're very serious.
You're only going to pick them up with massive, massive trials.
And those are very rarely conducted because they would be very expensive.
So the usual, people will be familiar with the evidence pyramid.
And the evidence pyramid goes like this.
You have, I mean, there are various different ones, but basically you have sort of the weakest sorts of evidence, like opinions and expert consensus and all of that at the bottom.
And then you get other sorts of evidence like case reports and case series, which are just descriptions of a particular patient or patients.
And then you have case control and cohort studies.
These are non-randomized studies.
They can be prospective where you've got a comparative where you give one group one treatment and another group no treatment and follow them up.
And then you get randomized controlled trials and the ones that are the beta standard are double blind, randomized controlled trials, but even double blind, randomized controlled trials have weaknesses.
And actually a double blind trial means that the patient doesn't know which treatment they're getting.
The clinician doesn't know which treatment they're getting.
In actual fact, what we really want these days is quadruple randomized controlled trials where the person assessing the evidence doesn't know and the person analyzing the evidence doesn't know.
And so even within a double-blind or triple-blind randomized controlled trial, if the group that holds the evidence has a particular outcome that they wish to convey, it's still possible for these trials to be manipulated.
They're not conflict-free, especially, you know, if they're conducted with their financial interests involved.
And then these trials are evaluated, grouped together in reviews, and reviews can be just general literature reviews, or they can be systematic reviews with meta-analyses, and those are considered The gold standard.
But if you think about all the people doing research on ivermectin, for example, there's evidence on all of these levels.
So this perimeter is full.
And to disregard all this evidence here, we've got all these experts around the world who are using ivermectin successfully.
to treat their patients and save lives.
So to say, well, that's actually meaningless.
What we're looking at is this little tip of the pyramid here to say, this is the only thing we're prepared to consider is really kind of crazy.
Wait, I want to pause you there for a second.
I think you've just said many things that are important and they will get lost if we don't unpack them a little bit.
I understood you to say that randomized control trials... First of all, I'm going to pat myself on the back here a little bit.
I said on the last live stream I did with Heather last Monday that Double blind randomized controlled trials were not blind from the perspective of those running the study.
That is to say they know what outcomes they want and that that means that these are not completely unbiased or not inherently so.
Sounds like you just said the same thing and you introduced me to a term I don't know which is quadruple blind, but I have in the past argued that an analysis is much better when somebody who has no stake at all is doing it.
Yeah, in the ideal world, you'd have the investigators of the trial conducting the trial and then handing their data over to a totally independent body for it to be analysed and reported.
All right, excellent.
Second thing I understood you to say was that the tip of the pyramid is actually not randomized control trial.
It is an analysis that looks at multiple kinds of evidence and integrates them, including but not limited to meta-analysis.
Is that fair?
No, the pyramid as it stands is systematic review with meta-analyses at the top.
And this is a real flaw, because, you know, you've got people doing their PhDs and doing very great work, and to say, well, if it's not a systematic review with meta-analysis, it's rubbish, and we're not even going to look at this type of study or evidence that you've done.
It's just, it makes a mockery, really, of academia and research in general.
So we're actually working on a different model, and it's actually not entirely novel.
I believe it has been Suggested before in a similar way and and that's really to look at evidence as as a as a circle as a pile or even as a donut with a center that that involves integration.
And actually I can show you, I've got a diagram.
Yeah, before you do that though, the other thing I heard you say that people I think will miss if we don't highlight it, is that in the case of ivermectin, you have the entire pyramid filled out, right?
We have evidence at all of those levels because it has been widely applied across the globe.
We have lots of people who have used it, have clinical experience, etc.
And so it is even more bizarre to limit one's understanding of the utility or lack thereof of the medicine to a single strata, right?
That's what we see, is people obsessed with a single layer of that pyramid.
So even if the pyramid is flawed and a donut would be better, It is odd that even within the pyramid structure, what we have is an obsession with a single strata, and it's one that obviously carries a built-in bias of its own, because it's a very expensive kind of test to run.
So it is basically Well, I hesitate to apply a judgment to it, but if one is obsessed with randomized controlled trial and one insists on these things as the only source of reliable evidence, that is effectively prioritizing new and therefore highly profitable medicines over existing repurposed ones where there's little profit to be made.
If you don't want to build that bias into the analysis, then the answer is you look at all the kinds of evidence.
If for some reason you want to stack the deck in favor of new drugs, the way to do it is to become obsessed with large-scale randomized controlled trials.
Yes.
And I must just say, you know, at the beginning of the pandemic, because obviously the need to have tools to treat and prevent COVID The UK's National Institute for Healthcare and Clinical Excellence produced a paper just to say, this is how we're going to look at evidence.
If there's a systematic review, we'll look at that evidence.
If there's a randomised controlled trial, we'll use that evidence to make our decisions.
If there's observational data, we'll use that.
So they sort of went down the list.
They said we'd use that, but if there wasn't that, we'd use randomised controlled trials.
If there weren't randomised controlled trials, we'd use observational studies.
If there weren't observational studies, we'd use expert opinion and consensus.
And they said they wouldn't bother to grade the certainty of the evidence or assess risk of bias, or assess the risk of bias of individual studies.
So they were just, you know, they were going to use what was available and make decisions based on that.
So to see how long they've taken, and in fact, they haven't even looked at ivermectin, but to see how long in general health authorities have taken to Evaluate and approve, I've mentioned, because it surely must be approved based on the body of evidence, is very surprising.
You know, I would like to know why.
All right, we will get back to that.
I want you to show us the model that you've been working on.
This is the improved model replacement, is it fair to say, for the evidence pyramid?
I think it would be more appropriate in the environment we're working in, where, as you say, there's just this huge Emphasis on these expensive randomized controlled trials, which is highly inappropriate given that we're in a pandemic.
And just generally, when there's huge amounts of evidence on generic medicines, which is being disregarded in favor of just the tip of the pyramid.
Is there any way you could enlarge that on your screen so we can see it a little better?
Marvelous.
Okay.
So, yeah, so this is what we're proposing is that, you know, evidence is looked at more as a pie and the center is where you integrate all the information that you receive, because it's not just numerical information.
There's also a lot of qualitative information and qualitative information is increasingly important in the World Health Organization's decision-making process.
The evidence on effects of a medicine is one of the criteria that's looked at when making a decision to recommend a medicine.
But one is also looking at what are people's values?
How do they value the outcome associated with this medicine?
And for example, if it's ivermectin, well, people really value the effect, value that it reduces deaths or reduces your chance of deteriorating or improves, you know, reduces transmission.
These are highly valued outcomes.
And one looks at Is it acceptable to people?
Well, we know Ivermectin is really acceptable because it's used 4 billion times and it's used around the world in so many different countries.
We know it's a highly acceptable and very, very feasible.
So is it a feasible medicine?
Well, we know.
So one looks at those types of, and that type of data comes from people's views and experiences and health professionals' views and experiences.
And as I say, that's increasingly important in And healthcare decision making and that data and information is being completely disregarded in the context of ivermectin.
But just looking at this, so this model, basically, we consider a pie, we have benefits on the one side and harms on the other or safety.
Now with regard to benefits or efficacy, Meta-analysis, so this quarter of the pie, the top quarter of the pie, we have meta-analysis, systematic reviews, and randomized controlled trials.
And they are the gold standard for determining whether a medicine is effective.
But they're not good at determining safety.
So for safety, we would have, so we would turn, so here we have observational real-world data and qualitative data.
That's sort of less important when you're looking at benefits.
You want to know, does this medicine work?
And when you're looking at safety, actually, what you want to know, you want to hear from the patients.
How do you feel?
What are your side effects?
You want qualitative data.
You want data from those pharmacovigilance databases where people report adverse events.
You want real world data and observational studies.
What are we seeing on the ground, in the field?
And evidence from the randomized controlled trials is really less Important.
And so with ivermectin, one has all this evidence because we've got real world data, we've got testimonials from patients, we've got experts around the world, doctors using the medicines, and so we know it works and that it's very safe.
And with, you know, if you look at A large randomized controlled trial done for a novel treatment that's just arrived on the block.
We don't have any expert consensus, any expert opinion on safety, any real world data or any patient experiences and views of the medicine.
All we have is perhaps something here in this slither of a pie over here.
for novel treatments and in many instances, nothing on safety, on long-term safety, particularly.
So for ivermectin, I'd say we have a whole pie.
We can integrate the evidence in the middle and say, when we look at the, we can see the big picture, this is a highly effective medicine in a range of contexts, in a range of settings, as well as in a range of of conditions from preventing illness to treating mild, moderate, and severe disease.
Whereas compared with new big pharma drugs that are coming in that are highly expensive, they do have a randomized trial, but they're just coming in with this sliver of evidence here.
And we have very little information on really how safe and acceptable and feasible and cost-effective these medicines are.
So let me continue to play student here.
The the.
In the case of something like a large scale randomized control trial, there are certain measures which are taken, but those trials are not going to be good at catching other kinds of phenomena, whereas
Clinical experience, for example, if doctors prescribe a drug and they find that some patients don't tolerate it well, that kind of knowledge will accumulate in their experience, their clinical experience, and so your new model here would integrate that information, whereas the obsession that we see with a single stratum in the pyramid We'll tend to exclude that information.
So basically you're arguing for a holistic approach that basically optimizes for good outcomes rather than for a proxy variable which may or may not be a good indicator of patient well-being or epidemiological effect.
Yes, yeah, and when you do a systematic review as well, you just chuck out all the studies that aren't randomized controlled trials, you know, and with ivermectin, for example, among others, there's many, many other studies.
So you literally get these, you do your literature search, and you're just really looking for the word random, and you know, and everything else just gets dismissed.
And there's some really fabulous studies, you know, a study done in Mexico by the Mexican government, where they gave people A little kit of ivermectin and zinc and paracetamol, I think it was, and as soon as they tested positive and they found that when they compared those data, and they did it in, I think it was 75,000 people or so,
And they compared it to hospitalizations.
And when they looked at hospitalizations before and after, they found that that approach had reduced hospitalizations by 50 to 70% or something like that.
And those kinds of really important and useful data and evidence are just completely ignored.
Yes, conspicuously so, because obviously that's a very dramatic effect, and it's not like one couldn't invent a reason that it could have happened otherwise, but it would have to be a pretty exotic explanation to account for that strength of signal, right?
It doesn't matter that the patients knew what they were being treated for.
The placebo effect can't keep you from needing hospitalization.
So, in light of that, a reasonable person who was doing an honest evaluation of evidence would have to admit that that was significant in spite of its not being randomized and placebo-controlled, whereas someone engaged in sophistry who wished not to acknowledge the effect of a drug in which there was no profit
might exclude it on the basis that the placebo effect might magically somehow interface with this, which of course there's no evidence for, and I don't know if you're unlikely to know that Heather and I covered the fact that the placebo effect, while it is significant, applies to subjective measures like pain and is not good at preventing people from needing hospitalization or keeping them from dying.
In fact, it doesn't do that at all.
Okay.
Good.
So is there more to say about an integrative approach and the value of it?
Well, I mean, one could just come up with a number of examples of other studies.
No, I think we've kind of covered it.
Great.
So then let me ask you a question or two.
There is evidence.
It has a certain amount of noise in it.
Your expertise involves compiling evidence of different kinds, discovering the signal within it, dealing with noisy data sets, and reaching a conclusion about patient well-being and potentially epidemiological effect of a given treatment.
What you are seeing with respect to the body of evidence that applies to ivermectin and its utility both as a treatment and a prophylactic for COVID, is that informing policy with respect to the use of ivermectin in a way that is familiar to you, or is there something unusual going on here?
No, it's really very unusual.
I've never seen such a huge body of evidence being ignored.
Many of the recommendations that are made on the guideline panels that I've been involved with are based on far less evidence, recommendations in favour.
Sometimes it's possible for there to be no clear evidence of effect even, but because an intervention is highly acceptable, desirable and feasible, A decision might be made in favor of it.
So it's very unusual for such a huge body of evidence to be ignored and to be calling for more large trials.
So you said you have never seen such a large body of evidence being ignored.
Yes.
That is a profound statement.
It's not that you have insignificant experience in this area.
You do this for a living, and you are seeing a body of evidence, and you've never seen one this large be ignored.
That's, I mean, in some sense, that sums it up, does it not?
But I'd also like to say, you know, I mean, as a medical doctor, you know, we do no harm.
You know, we do not take risks.
And I'm also a mother and I have three children, you know, and I wouldn't want any harm to come to them from a medicine that hasn't got a good safety profile.
So I think of, you know, when I think to myself, what do I feel about ivermectin?
Would I give it to my children?
Would I have wanted it in my cupboard?
Well, yes, the answer is yes.
And if I had a friend who was ill now, I would desperately want to get some ivermectin to them.
So, in combination with other, because Ivermectin is not the only effective treatment, we know that it works best in combination with other treatments that are also older, safe, over-the-counter medicines in many instances.
When I look at the evidence, I know that Ivermectin works.
It's confirmed over and over and over again.
And we know that it's very safe.
If I could just say, and many people who have heard me be interviewed before, you know, I always refer to the World Health Organization's VigiAccess database, because ever since January, when these newspaper reports started coming out about Adamectin being dangerous, potentially, or unsafe, I thought, well, let me follow it on the Pharmacovigilance database.
And since January, it's hardly changed.
The adverse events reporting has hardly changed, and it's being used by hundreds of thousands, if not millions of people at this very moment in time.
So one would expect if it was dangerous, one would see an increase.
One would see a signal the way we actually have seen with these remarkable vaccines that are being so widely deployed.
In actual fact, I have been tracking, because I was tracking Ivermectin, I started tracking the COVID-19 vaccines on the same database.
And now you can't even plot them on the same graph because the Adverse event reactions for ivermectin since 1992 are still in the region.
There are like 5,500 adverse drug reactions and billions of doses have been given.
And for the COVID-19 vaccinations, there's 1.3 million adverse drug reaction records on the WHO's Fiji Access Database.
Yes, and some very, very serious, including the last I looked, there were something like 10,000 deaths in the US.
And of course, some of those will be coincidence, but it's a very large number.
Do you want to say anything about that?
You've also done a meta-analysis.
Yeah, I am keeping my eye on it now.
There's 7,000 and some maybe 7,300 deaths on the WHO's database, there's 15,000 or so on the European database.
There's the VAERS database.
The UK has a fairly relatively small population and we've got 1,400 deaths recorded.
But it's not just the deaths.
I think it's the number of records.
And within those data, there are life-threatening and life-changing events.
So it could be that an event is reported But in actual fact, the sequelae, what comes after, are not reported because once it's reported, the report is done and kind of closed.
Whereas a person might go on to die or might go on to develop more serious events.
So, you know, my opinion on those data is that each and every one of those reports needs to be followed up.
Yes, especially in light of the fact that these vaccines did not go through a normal process, but are being administered under emergency use authorization, at least in the US.
I guess I confess I don't know what's going on in Britain or the rest of Europe.
Is it also the equivalent of an emergency use authorization?
Yes.
And so, just logically speaking, because it did not go through, or because it did go through a truncated process in which harms could have been spotted, it makes sense logically, and I believe also ethically, that is to say, according to the rules, that one would presume that adverse events were owing to the vaccines until proven otherwise.
And these pharmacovigilance databases are absolutely vital in the current context of unproven experimental vaccines.
So, you know, every report should be followed up.
It's an early warning system.
It's an early warning system.
Every analysis I have seen acknowledges that it undercounts, that there is a bias against reporting.
It's hard to calibrate how underreported it is, but it also, you're introducing me to something I didn't realize, which is that once an event is filed, it's closed.
So if somebody has a condition that gets worse, at the point they file it, It is effectively artificially held at whatever level it existed at that moment.
Is that right?
Yes.
Well, if you take a condition like Guillain-Barre syndrome, which is progressive paralysis, that is a fatal condition.
But if you look at, for example, the UK's Yellow Card database, it just shows you the number of cases and it shows you the number of fatalities.
But that number of fatalities is obviously going to go up over time, because we know that it is a fatal condition.
So, at the very least, we can say there's an alarming signal, and it really ought to have us engaged in extra scrutiny, and somehow, as with ivermectin, where you see what you're reporting is the largest body of evidence you've ever seen ignored, we have a very powerful signal of adverse events that is also largely being ignored.
It's a remarkable situation.
It is a remarkable situation.
The double standards is just so striking.
You know, when I've seen them, in fact, on the 5th of March, the FDA, your FDA, published concerns in the news about, concerns about the danger of, of ivermectin and using an unapproved medicine.
And clearly, you know, that, you know, the silence on the, The potential harms of the vaccines and the alerts that we're seeing on the early warning systems of these pharmacovigilance databases, the silence is deafening.
I would really like to hear something, some expression of concern from these regulatory bodies, because it certainly doesn't look like ivermectin is the medicine we need to be afraid of.
So, all right, you are a medical doctor, you are an expert data analyst, you are an external consultant for The Who, you have just reported what seems like The obvious response to the evidence, which is that we have a useful tool, we're not applying it in the case of ivermectin.
We have another tool that is potentially useful, but is showing an alarming signal that it is also doing harm to people.
And that is also being ignored.
The double standard between these two, right?
The way in which tiny harms that appear to owe to ivermectin or be correlated with it in some cases are being magnified what seems like wildly out of proportion in the same way that harms that are quite substantial that seem to be associated with the vaccine seem to be downplayed.
It all seems like there's a foregone conclusion that we haven't been told about that seems to be driving conclusions about what's safe and what isn't and what's effective and what isn't.
I mean, is there another interpretation?
No, but you know, I think if we're finding it difficult, I feel really sorry for people who don't understand the science of it and are trying to navigate and understand what's going on.
Yeah, I agree, and I think, unfortunately, what I've been saying is that there are two kinds of scientific consensus.
There's a natural consensus that arises when evidence increasingly points to something, and all of those who look at it come to the same conclusion.
And the example I've used is plate tectonics, which was originally an extremely controversial idea, but the evidence for it is so overwhelming that a consensus now exists.
All scientists I don't know if there are exceptions, but if there are any, it's a tiny few.
All scientists believe that the continents float, and plate tectonics is, even if it's imprecise, it is the correct, it is the accurate model.
And then there's a very unnatural kind of consensus.
It's a kind of consensus that is leapt to for reasons that have nothing to do with the evidence, and that's what I strongly believe is going on here, because at the very least, any honest broker would have to admit that there is substantial evidence in favor of ivermectin's effectiveness and in favor of significant harms that derive from the vaccines.
Anybody would have to admit that the evidence is there.
It doesn't mean that reasonable people couldn't disagree over what the balance of the evidence is, but to pretend that the evidence points to exactly the opposite of those conclusions in any clear way is preposterous, and yet That seems to be the consensus that you would imagine exists if you just simply tuned into the public discussion.
So it's a very bizarre disconnect.
All right.
Let me ask you this.
Two more questions in this sort of area before we move on.
One, This study that has been called into question most prominently in the Guardian article, the Guardian article which begins with a political attack claiming that those who have argued that ivermectin appears to be effective based on the evidence are somehow right-wing, which is a preposterous and obnoxious stigmatizing claim.
It is surprising that that article starts off with a political statement like that.
And it does seem to be a sign of the times that, you know, early treatment seems to be highly politicized.
I mean, I'm certainly not right-wing.
I don't put myself in any kind of political camp at all.
All right, so you're apolitical.
I would say, scientifically speaking, I'm apolitical.
What I say is it's natural for one's political beliefs to emerge from their science.
It is not natural or acceptable for one's political beliefs to inform one's science, and so I try to maintain that with great vigilance.
I follow the evidence where it goes, and if it tells me that something politically has to be true, then that's fine, but never the other way around.
I just like to stay away from labels generally.
I mean, I think being labelled, we're all very interesting and complex beings and I don't think labelling helps really in trying to resolve things.
It doesn't help, but in terms of understanding what is taking place in the public dialogue, it's at least worth noting that if I take something like the political compass test, which is I'm sure not a perfect test of anything, but it at least seems to be a decent measure of where one falls out politically.
I fall out very far left.
So the idea that this is somehow a right-wing phenomenon, that belief in this evidence or recognition of this evidence is a political phenomenon of the right, is simply at odds with the facts and preposterous.
And I think it's a tell that the Guardian article begins that way.
But let's come at this from another angle.
This is not the first study that has shown reasons for concern.
Would you like to talk at all about what you've seen in prior analyses?
Well, I mean, the nature of what we do with meta-analyses is to integrate evidence.
I don't really like to pick out any particular studies When what we're looking at is an overall effect.
So, you know, there have been other studies that have been very widely publicized and also have received a lot of criticism.
There was a study done in Colombia by López Medina that was published in JAMA, and it was widely hailed as being conclusive evidence that ivermectin didn't work, which clearly, as we've explained, is not the case when you Looking at pooling data.
And there were a number of limitations of that study, which made it really ridiculous to draw that conclusion that ivermectin didn't work.
And so, you know, what we have seen over the course of this evaluation of ivermectin is a preponderance of negative studies being published and positive studies languishing on pre-printed servers.
Now, usually there's something called Publication bias, and usually publication bias is in favor of positive studies, because journals have always liked to publish positive findings because they're more likely to sell journals, I guess.
People want good news, not bad news, and they don't want to hear that medicine doesn't work.
So it has been unusual to see negative studies being published in good journals and then and then being reaching the press very quickly and being highly shared, highly publicized on mainstream media to kind of emphasize erroneously that ivermectin doesn't work.
So that's so that was the one study and then the other the other review that also was quite widely kind of picked up by the media was a review by Roman et al.
And this review had some errors in it, which have still not been corrected, that pushed the analysis in favour of no difference with ivermectin, although it did show a difference and these findings were actually misinterpreted to draw the conclusion that there was no difference.
In any event, this flawed review Received prominence in the media and again was touted as suggesting that abomination doesn't work.
So yeah.
So wait, on that one.
So you've pointed to a couple of different anomalies.
One, so you say largest body of evidence you've ever seen being ignored.
That the publication bias in the case of ivermectin goes in a surprising direction.
In general, we all recognize that positive results are more dramatic, and therefore there's greater eagerness to publish them, and so that would be the typical bias.
In this case, we see exactly the opposite.
There seems to be a fervor for publishing negative evidence.
And not positive evidence, but then you point to this Roman et al.
study, and am I right that there was a clear, deeply embarrassing error made in the interpretation of the evidence, and then upon this being pointed out, they fixed the error in the analysis, but the conclusion of the paper remained unchanged?
I'm not going to go into the details, but it is a very flawed review, and we have asked for it to be retracted or corrected.
Retracted or corrected, and that has not happened yet?
No.
All right.
So people watching this surely are going to be scratching their heads.
One thing that people will scratch their heads over is who to trust.
And one thing that I think is important to recognize is that the incentive To bias anything in favor of ivermectin is quite low, because it's an out-of-patent drug.
There is not some group, as far as I'm aware, that has some financial incentive to support the idea that it works better than it actually does.
Whereas with respect to alternatives to ivermectin, whether it's Brand new drugs like Molnupiravir, or whether it's vaccines, there are conflicts of interest.
We can debate whether those conflicts of interest have implications for the evidence or its interpretation, but nonetheless, the conflicts of interest arise on one side of the equation very substantially, and they don't exist in any major way that I'm aware of, at least, on the other side.
Do you have any conflicts of interest that are relevant here?
No, I don't have any conflicts of interest.
I have no shares in any pharmaceutical companies or in Ivermectin and I have been working on this.
Initially it was for free because it wasn't a commissioned piece of work but now our work is funded by crowdfunding and I draw a modest amount from that crowdfund and I have done for the last two months.
So no, but also I just wanted to say, I think it's important also for people to know that I'm not anti-vaccination in general.
I have received all my vaccinations.
My children have received all their vaccinations, but we haven't received the COVID-19 vaccination.
And this is largely because I am concerned about the safety of these vaccines.
On the World Health Organization, database for a vaccine like tetanus, which I would have tomorrow if I stood on a rusty nail or got bitten, you know, something, I would, I would take that vaccine because since 1968 on the WHO's pharmacovigilance database, there's been 36 deaths and around 14,000 adverse events.
Whereas on the same database, there's been more than that number every day this year.
Um, against the COVID-19 vaccine.
So, you know, there are some vaccines that are very, very safe and that can be trusted.
And then there are these new vaccines, which I feel there's a strong signal, uh, that they lack the safety that we would like, um, for ourselves and our children.
So I will say the same thing.
My longtime viewers will know this, but I'm among the most vaccinated people I know.
My wife is, too, as a result of the fact that we did tropical field work and were vaccinated against some exotic things.
I would say I am very pro-vaccine in the abstract sense.
I think it's a very elegant technology that makes use of our endogenous immune system in a way that is hugely advantageous and has been a great boon to humanity.
I initially, when I first heard of the vaccines that were being developed, was hopeful that the vaccines would end the pandemic.
I assumed I would be vaccinated.
I was concerned that because the technologies being deployed were novel, that there might be long-term hazards.
And then, as short-term adverse events started to show up in large numbers, I became very concerned.
I am also not vaccinated.
I am on prophylactic ivermectin.
I am cautious about COVID.
I've been, I'm very concerned about the disease, and I've been concerned, Heather and I have been concerned from the beginning, and we've been proactive.
We were wearing masks when no one else was, so it is a substantial concern, but yes, I've arrived where you are.
As somebody who, in principle, favors vaccines, I'm alarmed by what I've seen.
It has altered my behavior.
I don't like the idea of being on any drug, including ivermectin, long term.
But from the point of view of having an active pandemic and vaccines that, to me, are alarming, it seems the better choice.
Yeah, well, you know, you just have to ask healthcare workers around the world, you know, are they taking ivermectin?
Would they take it?
And, you know, I think it's this Thousands of healthcare workers, doctors like Pia Khoury taking ivermectin to prevent COVID.
You know, I think it's worth, you know, I don't think you've got anything to lose, put it that way.
Yes, or a lot less to lose.
All right, so what else should we talk about?
We've talked about the concerns raised by this study and the anomalies that have been pointed out.
We've talked about different kinds of evidence and how they might be interpreted.
We've seen The value of meta-analysis and the fact that it has an elegant protocol for when a particular study shows signs of concern.
No, don't worry about it.
Fair enough.
What else should we talk about in terms of Well, yeah, well, I can tell you what, what our next steps are in the UK.
Well, good.
So we have a group called the British Ivermectin Recommendation Development Group.
It's actually an international group.
It's not just British doctors, and healthcare professionals, or scientists.
But we now have a number of affiliates around the world.
They've Other doctors groups, there's Italian groups, there's Canadian groups, there's Dutch and Philippines, groups in many different countries, Australia, who are wanting to use ivermectin-based protocols and are struggling to get the endorsement and approval from governments.
So we are collaborating more and more to produce protocols that can be used by health professionals who are wanting to provide early treatment.
And we are also trying to get the message to the public that COVID is treatable so they can feel less fearful.
You know, if they do get COVID, they can feel less fearful knowing that there are some treatment options available.
And also that there are options if you're not wanting to take the vaccine, that they are safe.
There's at least a good option.
to reduce your risk of getting COVID.
So we're doing, we have an initiative on the 24th of July, including all these affiliated groups to celebrate cautiously and a cautiously, a cautious celebration for World Advermectin Days.
So, this is an international initiative to raise awareness amongst the public that COVID is treatable and that avomectin is a very important component of treatment for COVID.
So, I will continue to beat The drum that I've been beating, which is Ivermectin is two drugs, it's two medicines.
One is a prophylactic, the other is a treatment, and the combination is very powerful, according to the evidence, in terms of preventing cases, and in cases that do occur, making those cases much less serious, especially when The medicine is given early, so I really believe that something strange is going on in this space.
It's reflected in what you've said about the large amount of evidence that's being ignored, but that if we had
public health officials that understood the evidence and were acting in the long-term interests of the public, then using the tools that we have at our disposal to maximal effect would render this pandemic much less serious quickly and potentially end it much sooner.
And the failure to do that is Conspicuous, bizarre, and unacceptable.
I mean, the idea that anybody would allow the pandemic to continue if we had tools that could in principle bring it to an end, I just find shocking.
Yeah, we've got something that, you know, that reduces transmission, prevents getting sick in the first place, and We've known about it for months and months.
So I think most concerning, you know, is that we see a number of people getting the vaccine, still getting COVID.
And in fact, many getting COVID-like illnesses, even long COVID-like illnesses.
And it's possible that they might also need ivermectin to treat these COVID-like illnesses.
So we're in a situation where So let me just go back to what's something that works against, prevents transmission, but also will work against variants because it has multiple mechanisms of action.
But it looks like we're also going to need it to help ameliorate symptoms amongst people who have COVID-like illness after vaccination.
So it seems like everybody needs ivermectin, whether you've been vaccinated or not, because the vaccines don't seem to work against variants and they seem to be associated with COVID-like illness.
Yes, this is a tremendously important point, right?
One hopes with respect to a vaccine that it will be perfect, meaning that it will prevent transmission completely.
The numbers that have been reported are reported as excellent, but another way of saying excellent is leaky.
That is to say a five percent failure rate of the vaccines to prevent COVID actually creates a selective environment in which variants are much more likely to be transmitted.
That is to say, those variants of the disease that are invisible to the immune system after the vaccines will have an advantage in the population in which people have been vaccinated in this way.
And so ivermectin is a backup for people who have been vaccinated and contracted the disease, or people who catch the disease from those who have been vaccinated and have had a breakthrough case.
It's the obvious tool to apply, and so our failure to apply it uh, irrespective of where you come down on the vaccines, the simple evidence of what they are and how useful they are suggests that we need a mechanism for treating cases that will be transmitted anyway.
I think, um, I mean the bottom line is that, you know, ivermectin is a sort of medicine everybody wants to have in their medicine cupboard, and it's a sort of medicine every doctor in an ICU or any hospital treating COVID patients would like to have readily available.
Well, I hope that people will listen to this podcast with an open mind, and especially for medical professionals who may have encountered what seems to be a propaganda campaign to discredit this very useful, not very profitable medication, will rethink their approach.
And in light of the Long-standing use of the drug and it's apparent.
Safety will try it.
What do you have to lose if you have a patient for whom you don't have an alternative trying the drug?
If it doesn't work, you can always discontinue it.
But it seems to me that people need to hear the evidence is clear.
We are lucky that we have as much information about the safety of this drug as we do, and that the signal is as good as it is with respect to safety.
And in light of that, you know, the Hippocratic Oath would seem to suggest applying the drug is the right thing to do for a patient for which you have no alternatives rather than withholding it.
Yeah.
All right.
Oh, sorry.
Medicine's a better word, yes.
Drug has all kinds of, you know, I guess drug covers positive things, negative things.
Medicine's better.
Yeah, I think, you know, ivermectin is a good old safe medicine.
Alright, excellent.
Anything else you'd like to say?
No, thanks very much, Brett.
This has been good and I appreciate the opportunity to have the chat with you.
Well, thank you very much, Dr. Teslori.
It has been a pleasure talking with you, and I certainly hope people will hear what it is you're trying to alert them to and take it very seriously.
