Why Was a Major Study on Ivermectin for COVID-19 Just Retracted?

May 10, 2022July 15, 2021 by Jack Lawrence

Questions about potential data fabrication, plagiarism, and ethical breaches led to the withdrawal of a study that formed a critical component of the pro-ivermectin case.

Editors note: Since publication Egyptian news outlets have reported that the Egyptian Minister of Higher Education has launched a review into the study. Also please note that in this piece we report that the paper’s authors listed their study participants as being aged 18-80 while having data for patients younger than this in what was stated to be their original data. We have since become aware that in earlier drafts of their paper the authors correctly reported the age range of 14-80. It is unclear why they changed this detail in their most recent update and this update does not materially change any of Grftr News’ other analysis. We still accept the authors may provide a satisfactory explanation for the questions raised and look forward to any response they might have.

Grftr News has discovered that the authors of an oft-cited study purporting to show a strong benefit when using ivermectin in the treatment of COVID-19 may have fabricated their data and plagiarised large portions of their paper. The preprint paper had provided the most positive evidence for using ivermectin to treat COVID-19 and had been cited in several other high-profile studies. After Grftr News contacted Research Square (the company hosting the preprint) alerting them of our findings, they swiftly removed the paper citing “ethical concerns.”

The study had formed a crucial piece of evidence in the pro-ivermectin case and its removal largely destroys (the current) scientific case for using the drug in COVID-19 care. Grftr News has reached out to the paper’s authors for comment but had not received a reply by the time of publication.

Ivermectin has long been used to treat parasitic worms in humans and sees frequent use for similar purposes in livestock and is very effective for both purposes. In contrast, little good evidence exists to justify its use in treating COVID-19. Both the Food and Drug Administration (FDA) and European Medicines Agency (EMA) advise against its use. Meanwhile, the World Health Organization only recommends its use within clinical trials.

To date, much of the evidence supporting ivermectin’s use in treating COVID-19 has been weak and inconsistent. A few studies have found a positive effect for ivermectin, with many others finding no effect.

In a June 2021 paper published in the journal Clinical Infectious Diseases summarising the current ivermectin research landscape, the authors concluded that it is “not a viable option to treat COVID-19 patients.” Other academics have argued that “research related to ivermectin in COVID-19 has serious methodological limitations resulting in very low certainty of the evidence.” To provide clarity, a team of researchers led by the University of Oxford is currently evaluating the effectiveness of using ivermectin to treat severe COVID-19 in a large clinical trial.

A Question of Evidence

On the 13th of November 2020, six researchers from two Egyptian universities uploaded a paper to the Research Square preprint server titled “Efficacy and Safety of Ivermectin for Treatment and prophylaxis of COVID-19 Pandemic.” Lead-authored by Dr Ahmed Elgazzar, a Professor Emeritus at the University of Benha, the paper claimed to represent the results of a multi-centre, 600-patient study evaluating the use of ivermectin in preventing and treating COVID-19. The authors claimed to have found that ivermectin significantly reduced both the number of deaths and the length of patient’s hospital stay compared to standard Egyptian treatment protocols. As if that wasn’t enough, the authors also claimed to show that ivermectin exhibited a substantial effect in preventing the onset of the disease in the first place.

Elgazzar is a former ‘chief editor’ of the Benha Medical Journal and a current editorial board member according to the journal’s website. He has previously authored several papers in the field of pulmonology.

The Elgazzar paper was met with interest by parts of the scientific community thanks to its astonishing results and because it was the largest trial on ivermectin to date. Despite never passing peer-review or being published in any scientific journal, the Elgazzar study went on to get cited in approximately 30 other studies, including several peer-reviewed ones in prominent journals. According to Gideon Meyerowitz-Katz, an epidemiologist at the University of Wollongong, doctors also used the results to justify treating “hundreds of thousands, if not millions of patients.”

From the start, the Elgazzar study also drew scepticism. A January 2021 briefing from the South African National Department of Health complained about a lack of specific details. It concluded that the paper suffered from a “high risk of bias due to reporting issues.” In an article on Medium, Meyerowitz-Katz previously blasted the paper for being internally contradictory and lacking crucial information, such as how the authors sorted patients into treatment groups.

Just a Series of Knife Slips?

After its initial publication, the Elgazzar preprint went through two further revisions. The purported authors published their third draft on the 28th of December 2020. It featured minor corrections and further statistical information but notably also included a link to what the authors stated was a copy of the original data, which Grftr News later obtained. (The paper’s withdrawal comes in the form of a ‘fourth’ revision).

When opening what the authors claim is their original data the first thing that any reader notices is that it’s remarkably complete. In many columns data for all patients are fully listed. The second thing the reader will likely notice is that the original data do not match the author’s public results. In three of the four study arms measuring patient death as an outcome, the numbers between the paper and original data differ.

In their paper, the authors claim that four out of 100 patients died in their standard treatment group for mild and moderate COVID-19. According to the original data they uploaded, the number was 0 (the same as the ivermectin treatment group). In their ivermectin treatment group for severe COVID-19, the authors claim two patients died – the number in the uploaded raw data is four. Grftr News put these findings to the authors however has not received any reply.

The original data provided by the authors suggest that efforts to randomise patients between different groups either failed or was not attempted – despite claims to the contrary by the authors. Every patient in the severe COVID-19 group receiving standard care was an ICU patient, while the patients with severe disease in the ivermectin group were mixed between wards and ICU. The experts Grftr News spoke to confirmed this is extremely unlikely to happen by chance.

Following these initial detections, Grftr News provided a copy of the data to science fraud expert Nick Brown and asked him to analyse it. Mere hours later, Brown had already conducted an extensive preliminary analysis and agreed to take a more in-depth look. Brown’s complete findings run for several pages and have been posted to his blog.

Wherever he looked, Brown found problems, from numbers containing non-numeric characters, confusion about date formats, and—most damningly of all—multiple incidences of data being copied between patients. In column after column data is duplicated exactly – “including the typo ‘coguh’ for ‘cough’ for multiple patients”. As Brown concludes in his blog post, “the chances of any one of these duplications occurring by chance, let alone all of them, are astronomical.”

It is also unlikely that these repetitions of data are due to an innocent copy and paste error when rearranging records in the file, as some columns contain identical patterns of data with minor adjustments to one or two numbers (possibly to make the fabrication less obvious).

Brown also discovered that many other numbers between the original data and the paper do not match. In some cases, numbers were off by only a couple per cent, while in others, the numbers were off by over 10%.

Grftr News also reached out to Kyle Sheldrick, a Sydney-based doctor and researcher who had been independently looking at the paper. Sheldrick explained that even when just looking at the results in the paper, problems emerged. For example, numbers that the authors provide for several standard deviations mentioned in tables in the paper are mathematically impossible given the range of numbers provided in the same table (standard deviations are a measure of variation in a group of data points)

Many of the patients who died appear to be duplicates. For example, according to the original data, there were ‘four’ patients with the initials NME, NEM, and NES (twice), who were all males aged 51 years old, all suffered from diarrhoea, had the same blood haemoglobin levels, were all diagnosed on the 22nd of May, and all died on the 29th of May 2020. They also all share identical values in at least four other data columns.

At least a further ten deceased patients also display evidence of being duplicated. As such, duplicates make up around half of the recorded deaths. Although much of the patient data is identical, minor changes exist, further proving that a simple copy and paste error cannot be the cause of the duplicates.

Sheldrick also argued that the completeness of data is further evidence of fabrication, noting that this is incredibly unlikely to happen in real-world conditions. In a Skype call with Grftr News, Sheldrick pointed out further statistical impossibilities, for example, noting that in the data column showing results for patient blood ferritin levels, “you’ve got at least 250 results there, and of those 250, only two or three of them end in three. And that’s just not possible. The probability of that is 2 in 10 billion.” (Nick Brown’s blog post explores this issue in more detail and finds that the trailing digits of many of the numerical values in the paper are even more implausible than that.)

If the dataset the authors provided is the one used for the paper then “this is obviously fraud”, Sheldrick explained, “there is no explanation for this other than fraud.” When pushed about whether alternative explanations are possible, Sheldrick replied, “I usually bend over backwards and stretch my incredulity; it’s the equivalent of ‘maybe the knife slipped multiple times’… but there is no [good] explanation for this. There’s no machine in the world that will have that sort of terminal digit bias.”

Brown, too, is convinced that the evidence points in the direction of fabrication. “It seems impossible to me that this data file, with its obvious cloning of substantial numbers of patient records and numerous other inconsistencies, contains a true record of the study of 600 patients that Elgazzar et al. claim to have conducted.”

After Grftr News made him aware of our findings Meyerowitz-Katz also wrote a further Medium article covering this situation and its consequences.

Grftr News put these findings to the authors of the paper but has received no response.

Plagiarism

Grftr News also detected what appears to be significant levels of plagiarism in the Elgazzar paper. With one or two minor exceptions, the entirety of the paper’s introduction appears to be copied from various sources, including several other studies, press releases, and letters to the editor from other journals. (Click here to see the evidence for yourself).

Where the copying is not verbatim, the author’s appear to have employed techniques more commonly used by students to disguise plagiarism, for example, by using synonyms or changing one or two words. This is how “severe acute respiratory syndrome” becomes “extreme intense respiratory syndrome” in one sentence in the paper, despite the fact that “Severe Acute Respiratory Syndrome” is part of the exact full name of COVID-19 (hence the name of the virus, SARS-CoV-2), and no scientist would paraphrase that sequence of four words regardless of how many times they had previously appeared in their article. Another example is “The coronavirus has been a known pathogen in animals since the early 1970s”, which in Elgazzar et al.’s preprint becomes “Coronavirus has been a recognised pathogen in animals in early 1960s”.

Grftr has reached out to the paper’s authors requesting a comment regarding this apparent plagiarism but has not received any reply.

Unethical Behaviour?

Seemingly not content with apparently potentially fabricating data and plagiarising their introduction, the authors decided to go for a hat-trick and engaged in some other ethically dubious behaviour too.

One sign of poor research design – though not fraud – was the author’s decision to only register their trial on a clinical trials registry after completing their study and publishing their first draft. Meyerowitz-Katz explained that while this is not optimal it’s still common. The purpose of registering a trial in advance is to avoid the authors changing the questions or analysis they perform once the trial is complete. Such behaviours would be considered bad practice, but not registering a trial isn’t proof that they occurred.

More problematic is the authors’ decision to provide conflicting information about their trial start date. In their trial registry information, within the paper, and in replies to comments on the paper, the authors claim to have received ethical approval and commenced the trial on the 8th of June 2020. However, according to their original data, the authors recruited and treated several patients before this date. Moreover, almost half of the patients who died during the trial died before this date. Both Meyerowitz-Katz and Sheldrick confirmed that this is a problem. If the authors started their study before they had ethical approval, this would be a major ethics violation. Additionally, the authors claim to have conducted their trial on 18-80-year-olds, but the original data contains records for four patients younger than 18. Update: Grftr News has learned that in earlier versions of the paper the authors correctly stated patients were aged 14-80. It is unclear why this was changed in their most recent revision.

Grftr News approached the Benha University Research Ethics Committee and the paper’s authors with several questions regarding these aspects of the study but has received no response.

Grftr News approached Research Square, the company which controls the preprint server the Elgazzar paper had been published on for comment. A day later, on the 14th of July, a notice appeared on the study stating in full, “Research Square has withdrawn this preprint due to ethical concerns.” Michele Avissar-Whiting, Research Square’s Editor in Chief, later explained to Grftr News that although previous versions of the preprint currently remain online, the paper’s permanent digital object identifier (DOI) would link to the withdrawal notice.

Now What?

“That almost entirely reverses any argument for using ivermectin,” Meyerowitz-Katz explained in response to Brown and Sheldrick’s findings because “the Elgazzar study is the single biggest piece of research” conducted on ivermectin.

He was also keen to rerun the results of two recent meta-analyses that found an overall positive effect for ivermectin, this time without the Elgazzar data. (A meta-analysis is a type of scientific paper that summarises the existing evidence on a specific topic by aggregating the results of smaller trials). After excluding the data from the Elgazzar study, he found that the effect for ivermectin drops significantly with no discernible effect on severe disease. Meyerowitz-Katz later also reran the analysis while excluding an additional poor-quality study and found that after this ivermectin showed no effect in treating COVID-19.

Asked whether he thought ivermectin may still turn out to have some benefits, Meyerowitz-Katz granted that better-designed trials might find some benefit. Adding “but it’s extraordinarily unlikely that ivermectin will have the sort of benefits being proposed by the groups who want everyone to use it.”

Sheldrick also had strong words to say about the study, “ivermectin’s not a drug with no side effects. And if there are doctors around the world who were tricked by this research and they go, and they prescribe a drug that doesn’t work, or worse is harmful, then the result is patients die. If patients take this instead of treatments that work, then the result is patients die.”

The experts Grftr News spoke to agree that the findings made concerning the paper and its data cannot easily be waved away. The Elgazzar paper, its data, and its authors are fundamentally suspect. As Sheldrick explained, “there really isn’t any process now by which this can be corrected. Once a group or at least members of a group have been willing to just wholesale fabricate data, if they now come out with a second data set that’s different, how can we trust that?”

Something Rotten in the State of Science?

Though the fabrication and plagiarism are egregious and a severe breach of scientific integrity, the paper’s damage would have been more confined were it not for the efforts of a cadre of ivermectin hypers.

Contrary to the claims of conspiracy-minded thinkers, ivermectin has seen support from a range of influential political figures. In December 2020, Dr Pierre Kory, a leading figure in the Frontline Covid-19 Critical Care organisation, which heavily promotes ivermectin, was invited to appear in front of the United States Homeland Security and Governmental Affairs Committee. Kory included a reference to the Elgazzar paper in his testimony. He also later included the study in a review on ivermectin that he helped author. Ivermectin has also been promoted several times in both Houses of the Parliament of the United Kingdom (1, 2, 3, 4, 5) and at least three times in the Australian House of Representatives by MP Craig Kelly (1,2,3).

A cottage industry of pro-ivermectin advocacy groups has also sprung up, each boasting of their founders medical and scientific credentials. Along with the FLCCC, another prominent organisation has been the British Ivermectin Recommendation Development (BIRD) Group. Members of this group also recently published a study citing the Elgazzar paper. Together these organisations and their supporters have made a big splash and have drawn headlines around the world. It is unknown how these groups will now respond to the news of the Elgazzar study’s withdrawal.

Even if the paper’s authors end up providing an innocent explanation for all this it would be puzzling why it took them so long to notice their error. Whether the final story is one of purposeful fabrication or a series of escalating mistakes involving training or test datasets, this research group has still screwed up in a big way.

Although science trends towards self-correction, something is clearly broken in a system that can allow a study as full of problems as the Elgazzar paper to run unchallenged for seven months. Thousands of highly educated scientists, doctors, pharmacists, and at least four major medicines regulators missed a fraud so apparent that it might as well have come with a flashing neon sign. That this all happened amid an ongoing global health crisis of epic proportions is all the more terrifying. For those reading this article, its findings may serve as a wake-up call. For those who died after taking a medication now shown to be even more lacking in positive evidence, it’s too late. Science has corrected, but at what cost?

—

If Grftr News hears anything from the authors of the Elgazzar study we will update our piece with their comments. While we have total confidence in our findings we accept that the authors may still provide evidence showing that the issues we have raised are down to good faith mistakes rather than any malicious motive.

Please also note that Grftr News does not and has never received any funding or income from any pharmaceutical company or lobbying group. We also do not run any advertising or work with or on the behalf of any other company, group, or organisation. Our only funding comes from our wonderful patrons on Patreon. If you would like to join their numbers you may do so here or leave us a one-time donation on Paypal here: https://paypal.me/gftrnews

Jack Lawrence

Jack Lawrence is the founder of Grftr.news. He is currently pursuing a master’s degree in biomedical science and spends his free time writing about and researching misinformation and bad science. He also runs the @TimPoolClips Twitter project.