The coronavirus disease 2019 (COVID-19) pandemic has placed the scientific and research communities under extraordinary pressure, to which they have responded with exceptional vigor and speed. This desire to quickly find safe and effective treatments may also lead to relaxed standards of data generation and interpretation, which may have undesirable downstream effects. The recent publication of a study evaluating hydroxychloroquine (HCQ) in COVID-19 is a useful test case, highlighting the challenges of conducting research during a pandemic.

A scientific rationale existed for investigating HCQ in COVID-19. Preclinical data suggested that the antimalarials HCQ and chloroquine have in vitro antiviral activity against severe acute respiratory syndrome–coronavirus 2 (SARS–CoV-2) (1–3). Antimalarials are also inexpensive, are widely available, and have an acceptable short-term safety profile. In a nonrandomized study, Gautret and colleagues (4) reported a higher frequency of SARS–CoV-2 clearance from the nasopharynx after 6 days of treatment with HCQ, plus azithromycin (AZM) if deemed necessary, versus an untreated control group (14 of 20 patients [70%] vs. 2 of 16 patients [13%]; P < 0.001). Given the urgency of the situation, some limitations of this study may be acceptable, including the small sample size, use of an unvalidated surrogate end point, and lack of randomization or blinding. However, other methodological flaws also noted by others (5) may affect the validity of the findings, even in the current setting, where an efficacious treatment is desperately needed.

First, potentially substantial confounders may explain the findings. The HCQ + AZM treatment group was recruited from a single center. Instead of excluding patients who declined treatment, the researchers assigned them to the control group. The remainder of the control group was recruited from other centers that could not contribute to the treatment group. This introduces the potential for baseline confounding and different treatment regimens at different institutions. In addition, patients in the HCQ + AZM group had lower viral loads at the time of treatment initiation compared with the control and HCQ groups, and so may have been at a later phase of infection. All patients who received HCQ + AZM had a SARS–CoV-2 baseline cycle threshold (Ct) greater than 22 on polymerase chain reaction (PCR). Of the 5 patients receiving HCQ who had a baseline Ct of 22 or less on PCR (that is, higher viral burden), 4 still had detectable virus at day 6. Of the 9 patients with a baseline Ct greater than 22 on PCR, only 2 had detectable virus at day 6. Thus, another explanation is that the baseline viral load, not therapy with HCQ + AZM, affects viral load at day 6.

This problem may have been further exacerbated by issues with data measurement and imputation. Sites other than Marseille did not perform daily PCR testing, creating gaps in PCR data for control group patients that were later imputed with values from other days. Consequently, 75% (12 of 16 patients) of the control group lacked at least 1 PCR result, including 44% (7 of 16 patients) who were not tested on at least 4 of the 7 possible days. In the HCQ + AZM group, only 20% (4 of 20 patients) were missing at least 1 PCR result and none were missing more than 2. In the control group, 38% of the PCR data were imputed versus only 5% in the treatment group.

Second, the handling of patients who were lost to follow-up also raises serious questions about scientific validity. Only 20 of 26 patients in the treatment group were included in the analysis despite meeting baseline eligibility criteria. Six patients were excluded because day 6 PCR data were missing, owing to early treatment cessation due to nausea (n = 1), hospital discharge (n = 1), intensive care unit transfer (n = 3), and death (n = 1). Therefore, patients who had the most serious and clinically relevant outcomes, including intensive care unit transfer and death, were excluded from analysis. These patients had treatment failure and should have been analyzed as such. We strongly agree with others that adequate follow-up of patients with relevant outcomes should be attempted (6).

Despite the study's substantial limitations, a simplification and probable overinterpretation of these findings was rapidly disseminated by the lay press and amplified on social media, ultimately endorsed by many government and institutional leaders. Public interest in HCQ rapidly grew (Figure). The study's findings were extrapolated to include the use of HCQ to prevent COVID-19 infection or as postexposure prophylaxis, indications for which there are currently no direct supporting data. Despite promising in vitro data for influenza (7, 8), HCQ failed to prevent infection in a randomized, placebo-controlled, double-blind trial (9). Efforts to understand the clinical efficacy of HCQ as postexposure prophylaxis are under way (https://clinicaltrials.gov/ct2/show/NCT04308668), which should yield important insight into this issue.

A major consequence has been an inadequate supply of HCQ for patients in whom efficacy is established. Hydroxychloroquine is an essential treatment of rheumatoid arthritis and of systemic lupus erythematosus, reducing flares and preventing organ damage in the latter disease (10). Pharmacies have reported shortages of antimalarials (www.washingtonpost.com/business/2020/03/20/hospitals-doctors-are-wiping-out-supplies-an-unproven-coronavirus-treatment), and patients with rheumatic diseases have had difficulty obtaining prescription refills. Several major medical organizations released a joint statement regarding the HCQ shortage (www.lupus.org/s3fs-public/pdf/Joint-Statement-on-HCQ-LFA-ACR-AADA-AF.pdf), warning of possible dire consequences for patients with rheumatic diseases. Hydroxychloroquine shortages could place these patients at risk for severe and even life-threatening flares; some may require hospitalization when hospitals are already at capacity. Until reliable evidence is generated and adequate supply chains have been put in place, rational use of HCQ in patients with COVID-19 must be emphasized, such as use in investigational studies.

In critical situations, large randomized controlled trials are not always feasible or ethical, and critically ill patients may need to be treated empirically during times of uncertainty. However, it is our responsibility as clinicians, researchers, and patient partners to promote proper and rigorous interpretation of results, particularly in our interactions with the nonscientific community. We must consider the societal implications of published work in these unprecedented times.

There is enough rationale to justify the continued investigation of the efficacy and safety of HCQ in hospitalized patients with COVID-19. It is critical to reiterate that although viral clearance is important, clinical outcomes are much more relevant to patients. There currently are no data to recommend the use of HCQ as prophylaxis for COVID-19, although we eagerly await data from trials under way. Thus, we discourage its off-label use until justified and supply is bolstered. The HCQ shortage not only will limit availability to patients with COVID-19 if efficacy is truly established but also represents a real risk to patients with rheumatic diseases who depend on HCQ for their survival.

For more information on the alliance, including contact details of the steering committee and regional leaders, visit https://rheum-covid.org/about.

Philip Robinson, MBChB, PhD, FRACP, MAICD (Chair, Governance, Policy); Jinoos Yazdany, MD, MPH (Vice-Chair, Real-world Data Infrastructure, Registry and IRB/Ethics); Paul Sufka, MD (Technology and Marketing Lead); Rebecca Grainger, MBChB (Dstn), BMedSci (Dstn) MIsntD, CHIA, FRACP, FACHI, PhD (Literature Review Co-Lead); Zach Wallace, MD, MSc (Literature Review Co-Lead); Organizational Liaison and Media); Emily Sirotich (Patient Engagement Lead); Jean Liew, MD (Administrative Lead); Jonathan Hausmann, MD (Patient Registries Collaboration Lead); Pedro Machado, MD (European Lead).

Australia: David Liew

Brazil: Claudia Marques

Canada: Diane LaCaille, Sindhu Johnson, Keshini Devakandan, Carter Thorne

China: Mengtao Li

France: Francis Berenbaum

Germany: Johannes Knitza, Peter Korsten

Ireland: Richard Conway

Latvia: Bulina Inita

Mexico: Erick Adrian Zamora Tehozol

New Zealand: Rebecca Grainger

Peru: Manuel Ugarte-Gil

Saudi Arabia: Ibrahim Almaghlouth

South Africa: Kate Webb

United Kingdom: Taryn Youngstein, Pedro Machado, Richard Beesley, Kimme Hyrich

United States: Adam Kilian, Maria Danila, Isabelle Amigues, Eugenia Chock, Michael Putman, Laura Lewandowski, Jon Hausmann, Maximilian Konig, Beth Wallace, Reema Syed, Sebastian Sattui, Arundathi Jayatilleke, Jean Liew, Namrata Singh, Aarat Patel, Katherine Wysham, Alí Duarte, Marc Nolan