Making Sense of Conflicting Studies on COVID-19 Drug Remdesivir

Treating any infectious disease is a high wire act—doctors must balance the risks and benefits of therapies with the risks of the disease. And those stakes are even higher for a new disease that doesn’t yet have a playbook for physicians to follow.

This summer, the U.S. Food and Drug Administration gave the drug remdesivir emergency use authorization for treating any patients hospitalized with COVID-19. But on Oct.15, researchers working on the World Health Organization’s (WHO) Solidarity trial published a preliminary report showing that nearly 3,000 people receiving the treatment were not more likely to survive infection with SARS-CoV-2 than those receiving standard of care.

The study involved more than 11,000 people hospitalized with COVID-19 who were randomly assigned to receive one of four different therapies—remdesivir, hydroxychloroquine, lopinavir, or interferon—for their infection, or standard of care. After following the patients for about a month, the researchers found that none of the drugs reduced mortality or cut down on hospital stays.

The remdesivir findings are in direct contrast to a report published Oct. 8 in the New England Journal of Medicine by a team led by researchers at the National Institute of Allergy and Infectious Diseases (NIAID), which found that remdesivir helped hospitalized COVID-19 patients to recover faster compared to placebo. While that study also did not find a statistically significant effect on mortality, the shorter hospital stays and shorter time on oxygen and ventilators suggest that the drug might provide some benefit for patients in overcoming their infections.

Remdesivir’s maker, Gilead, took issue with the findings, and in a statement quickly criticized the design of the study and the fact that the findings were preliminary and not published in a peer-reviewed journal: “The emerging data appear inconsistent with more robust evidence from multiple randomized, controlled studies published in peer-reviewed journals validating the clinical benefit of Veklury (remdesivir).”

So what do the conflicting results mean? Does remdesivir benefit COVID-19 patients or not? Experts are just as divided as the contrary results. “Here are the facts of this study,” Dr. Andre Kalil, professor of internal medicine at the University of Nebraska who was among the first to treat patients coming off the Diamond Princess cruise ship with remdesivir, says of the WHO Solidarity trial, “No data monitoring, no placebo, no double-blinding, no diagnostic confirmation of infection, no timing of symptom duration before treatment initiation, unknown baseline physiological severity, unknown supportive care provided, unknown health care capacity status of enrolling sites, and a large amount of missing data. Poor quality study design cannot be fixed by a large sample size, no matter how large it is.”

Dr. Eric Topol, director and founder of Scripps Research Translational Institute, on the other hand, notes that compared to the NEJM study, “This [Solidarity] is a much larger trial, done throughout the world at 400 hospitals. I think that gives this trial a little more weight.”

It turns out that the findings may not be as at-odds as they appear. There isn’t much detailed information on the hospitalized patients included in Solidarity, including, as Kalil notes, how long they were sick, why they were hospitalized and whether they had additional health conditions that could make them more vulnerable to severe COVID-19 disease. Dr. Roger Shapiro, associate professor of immunology and infectious diseases at Harvard T. H. Chan School of Public Health, and a practicing infectious disease physician, pointed out that, unlike the NIAID study, the Solidarity study did not break down people who needed oxygen by how much additional oxygen they needed. The Solidarity study grouped people by whether they did not need supplemental oxygen, whether they needed it or if they were on ventilators.

The difference could be important, as those needing less oxygen may be earlier in their infection than those needing more. Indeed, in the NIAID study, the scientists did look at the so-called low-flow and high-flow groups, and found that remdesivir provided more benefit for those needing less oxygen. And Shapiro notes that if the two groups in the NIAID study are combined, and treated as a single group like these patients were in Solidarity, the responses to remdesivir are pretty similar.

That suggests that it’s important to know how sick people are to determine how they might respond to remdesivir. Experts now believe that based on how remdesivir works, and how COVID-19 infections march along, the drug may be more effective earlier and soon after infection rather than later. Because remdesivir is an antiviral drug, it can thwart SARS-CoV-2’s ability to copy its genome and produce more virus. So it likely will have a more potent effect when there is less virus circulating during the early days of infection.

It’s also not clear from the preliminary Solidarity report how sick patients were when they were hospitalized. And because the trial involved 405 hospitals in 30 countries, the criteria for hospitalizing patients might have differed. “We have a lot of unknowns right now about who the patients in Solidarity were,” says Shapiro. “It’s really important to think about the diversity of what ‘hospitalization’ means in this study. In some places, people were hospitalized when they were near death’s door. In other places there may have been a tendency to hospitalize COVID-19 patients in order to isolate them, before they might have met certain criteria for requiring it.”

If, for example, most of the patients in Solidarity were severely ill and further along in their disease, then they might not have benefited as much from an antiviral like remdesivir. And while the Solidarity data show that only 9% of participants in the study were on ventilators, that may, in part, reflect shortages in ventilators or hospital beds for those needing mechanical ventilation and not a true reading of how many people had severe enough disease to need the machines. “It raises the question about severity of illness, and how long people were sick before they started the study,” says Shapiro.

It’s also worth noting that Solidarity enrolled patients from March to October, over which time standards of care for COVID-19 changed dramatically. At the start of the pandemic, doctors had little to offer patients and were quick to intubate people and put them on ventilators if they complained of breathing problems. Now, doctors are advising patients to lie on their stomachs to reduce the burden on the lungs and know from other studies that ventilation can worsen symptoms rather than improve them.

“We have made so many changes in the last six months in the supportive care we provide for patients, and they are all incremental,” says Dr. Aruna Subramanian, professor of medicine at Stanford University who led a study sponsored by Gilead and released in April showing that remdesivir over a five-day period was as effective as a 10-day course. “Looking at one thing in isolation [like remdesivir] and saying that it doesn’t reduce mortality doesn’t make sense to me. It’s like the blind man feeling the elephant’s leg or trunk or tail and not being able to see the whole picture.”

Topol notes that given the conflicting results, it’s more likely that remdesivir may have a modest effect in mitigating COVID-19. “The Solidarity trial study is very short, and provides limited data,” he says. “Do I hope to see more data, yes. But I don’t know if that is going to change how it’s interpreted.”

The Solidarity study has not been peer reviewed, which means experts have not evaluated the methods and conclusions of the study yet. And the study was open-label, meaning patients knew they were receiving the drug. While the study authors argue this likely did not have an effect on the results given the size of the study, it’s not the scientific gold standard for evaluating drugs. The NIAID study, on the other hand, randomly assigned people to remdesivir or placebo and neither the doctors nor the patients knew which they received.

“I don’t think this study should be the death knell for remdesivir,” says Shapiro. “When we have a placebo controlled study like [the one from NIAID] that shows us a strong signal, you can’t poke a hole in that. That’s our gold standard. The size of Solidarity does give you pause but the signal we are seeing with the NIAID study is hard to explain away. We just can’t take Solidarity and say it supersedes what we saw with the [NIAID] trial. We just can’t do that because the quality of the [NIAID] trial is better.”

For Subramanian, “[Solidarity] doesn’t really change my standard of care at this time. I know based on our data that remdesivir helps people to leave the hospital quicker. And we are going into flu season when hospital beds are often very critical. Getting people better faster, and off of oxygen and home faster, is a very important outcome.”

Contact us at letters@time.com.