FormalPara Key Summary Points

Multiple experimental/compassionate drugs were and are being administered to patients with COVID-19.

During the first pandemic wave, patients with severe COVID-19 were more likely to receive more experimental/compassionate drugs.

Experimental/compassionate treatment was unrelated to survival but may have been associated with longer duration of hospital stay.

When multiple experimental/compassionate medications are administered to the same patient, the true effect of any single drug remains questionable even after adjustment for receipt of additional treatment.

Introduction

A large number of drugs have been proposed as potential cures for COVID-19 and many of these drugs are being tested in medical centers. This unique collaboration between medical centers and researchers stems from the desperate hope that at least one of the many drugs tested will mitigate disease severity among the patients being treated [1]. However, this phenomenon may also create a situation in which patients receive more than one experimental drug. For example, in one multi-center study of remdesivir, patients were included despite concomitant use of lopinavir/ritonavir, interferon alfa-2b, and corticosteroids [2]. Further examples can be found in recent literature. The dexamethasone trial of the RECOVERY group that examined the use of corticosteroids in COVID-19 did not exclude patients who were treated with other experimental drugs, although the reported prevalence of use was similar in the different trial arms [3]. In the final report of the effectiveness of remdesivir in the treatment of COVID-19, overall, 23% of patients included in the study received corticosteroids and 35.6% received hydroxychloroquine, along with other experimental drugs [4].

An editorial has highlighted the potential harm in reporting the same patients in different articles. The issue of mixed treatment may also be a major research setback. When multiple treatments are administered concomitantly it is nearly impossible to isolate the effect of a single treatment. Few studies have adjusted for the concomitant use of multiple medications in patients with SARS-CoV-2. However, when such control was achieved, prior drug effects often disappeared. Chan et al. showed that, in patients who received lopinavir/ritonavir for treatment of severe acute respiratory syndrome, no drug effect was observed, after matching for the use of pulse steroids among other characteristics (age, sex, co-morbidity, maximal LDH level) [5].

A larger sample size is required to elicit the effect of a specific drug when treatments are mixed. However, even large randomized controlled studies may not be able to control for confounding if the calculated sample size is not recruited. Studies terminated early because of temporary outbreak control can therefore provide only limited information. A typical example is the early remdesivir study in which the authors planned to recruit 453 patients in order to achieve 80% power under a one-sided type I error of 2.5%. However, analysis was conducted on only 236 patients, which reduced the statistical power of study to 58% (as calculated by the authors) [2]. Additional studies were required in order to elicit that the drug actually had little effect on major patient outcomes [4, 6]. Control of the outbreak is a much wished for outcome, and multiple measures have been implemented globally to ensure that systems are not overwhelmed with critically ill patients. Moreover, if the number of critically ill patients increases above a certain tipping point, the level of care would need to be reduced and little value may be gained by specific drug treatments provided to these patients. Conversely, if the control measures being implemented are effective there is little likelihood of recruiting a sufficient number of patients in individual trials. In case of multiple small studies, data pooling is often the solution. However, mixed treatments are also a problem in this context as they preclude data pooling and mandate obtainment of individual-level patient data [7].

We therefore aimed to study whether critically ill patients receive more experimental/compassionate medications than do non-critically ill patients, how many experimental and compassionate drugs were administered to an average COVID-19 patient during the first pandemic wave, and whether outcomes [survival and length of hospital stay (LOS)] were related to the number of experimental/compassionate medications received. We hypothesized that the more severe the patient seemed in terms of their respiratory disease, the more likely they would be to receive multiple medications, and that survival is not associated with medication when adjusted for disease severity.

Methods

Study Design

This retrospective, single-center analysis of observational data collected in real time was conducted after receipt of Institutional Review Board rapid approval by the Shaare Zedek Medical Center Institutional Review Board (0119–20-SZMC). The need for informed consent was waived in order to avoid inclusion bias in favor of less critically ill patients and because of the observational nature of the study.

Clinical Setting

The Shaare Zedek Medical Center (SZMC) is a tertiary hospital located in the center of Jerusalem. Before the pandemic, the hospital had 1000 beds (about 850 for adults) and 14 general ICU beds. Intensive care is provided by a team of board-certified doctors and nurses. Since the pandemic onset in March 2020, the ICU team had been divided, rotating between the general ICU, the COVID ICU, and COVID intermediate care. Lacking strong evidence, all treatment decisions were made through team discussions and consultations, with the final adjudication made by the most senior doctor rounding each day. Limitation of care is illegal in Israel unless the patient presents a formal advance directive that has been signed by an attorney. Moreover, advance directives are very rare in Jerusalem, and as during the first pandemic wave resources were not overwhelmed at the SZMC and the likelihood of patient survival was unknown, limiting care was not considered an option.

Objectives

Our main objective was to assess the correlation between respiratory disease illness severity (defined by the maximal respiratory support received during admission) and the number of experimental/compassionate medications received. Secondary objectives were to describe the number of experimental and compassionate drugs that were administered to an average COVID-19 patient, and to study whether outcomes [namely survival to hospital discharge and LOS), were related to the number of experimental/compassionate medications received.

Participants and Inclusion–Exclusion Criteria

We included all adult patients (age ≥ 18 years) admitted to the SZMC departments and ICU between 1 March and 17 April 17 with laboratory confirmed SARS-CoV-2 infection (based on PCR testing of nasopharyngeal swab specimens) and respiratory symptoms suggestive of COVID-19 disease. We excluded patients admitted to hospital for conditions unrelated to SARS-CoV-2 infection (e.g., labor, trauma, chemotherapy, etc.) who were incidentally found to be infected in routine screening.

Data and Sources

All data were extracted from the electronic medical records during admission (hot pursuit). The data collected included patient demographics (e.g., sex, age) and comorbidities including body mass index (BMI). Data regarding comorbidities were also validated in retrospect to ensure full coverage. Also collected were admission vital signs [heart rate, blood pressure, temperature, O2 saturation (O2Sat)] and admission laboratory values [white blood cell count, lymphocyte count, hematocrit, C-reactive protein (CRP) and ferritin].

We considered medications as experimental when administered within the framework of a clinical trial. Medications were considered compassionate when administered to a patient with life-threatening COVID-19 despite lack of clear proof of benefit when no comparable or satisfactory alternative therapy options were available [8]. We identified the following compassionate\experimental treatments that may have been administered to patients during the study period: azithromycin, hydroxychloroquine, corticosteroids, remdesivir, convalescent plasma, lopinavir, opaganib, and tocilizumab.

Management of Quantitative Variables

A BMI greater than 30 was classified as obese.

We classified disease severity based on the most invasive method of oxygen support provided during admission [no support, nasal cannula, regular face mask, high flow nasal cannula/BiPAP, invasive mechanical ventilation and extra-corporeal membrane oxygenation (ECMO)]. Invasive mechanical ventilation and ECMO were pooled due to the small number of patients treated with ECMO.

For the purpose of analysis, we also defined two groups; critically ill patients (i.e., those supported with high flow nasal cannula, continuous positive airway pressure, mechanical ventilation or ECMO) and non-critically ill (the rest) for performing group comparisons.

Outcomes

The primary outcome measure was the relation between patient severity (defined either the maximal respiratory support received during admission or by classification as critically ill/not critically ill) and the number of experimental/compassionate medications received.

Secondary outcomes included: (1) a description of the number of experimental and compassionate drugs that were administered to COVID-19 patients overall, and (2) the unadjusted and adjusted relationships between patient severity (as assessed by clinical and laboratory parameters), the number of experimental/compassionate medications received and patient outcomes [survival to hospital discharge and length of stay (LOS)]. For the latter, we studied the association between patient severity, number of experimental/compassionate medications received, and survival to hospital discharge (unadjusted and adjusted analyses), and the association between the number of experimental/compassionate medications received and LOS (unadjusted analysis).

Address of Bias

Admission bias was less common in the early stages of the pandemic when disease progression and patient outcomes were not clear; at this time, most patients were admitted once diagnosed as symptomatic. Documentation bias was minimized by implementing real-time data collection for research purposes. All data were performed by a single researcher (O.A.) throughout patient admission (i.e., hot pursuit methodology). We documented and present missing data. For the variables considered critical to this study (patient age, sex, maximal respiratory support received during admission, experimental/compassionate medications, and outcome) there were no missing data. Other variables with a large proportion of missing data were excluded from the analysis.

Study Size

Sample size estimation was based on the expected difference in proportions of patients receiving two or more compassionate/experimental drugs among critically and non-critically ill patients. We assumed 60% among critically ill patients and 10–35% among non-critically ill patients because of the level of uncertainty inherent to the treatment of SARS-CoV-2. Overall, 40 critically ill patients and 250 non-critically ill patients would need to be studied to prove that the difference between the groups is statistically significant at a one-sided level of 5% with a statistical power ranging between > 99.9% (for 10%) and 80% (for 35%).

Statistical Methods

Descriptive statistics were first used for the study population and their outcomes. These included frequencies and percentages for categorical variables and means (± standard deviations) as well as medians, ranges, and interquartile ranges for continuous variables.

For comparisons, non-parametric tests were applied in case of non-normal variable distribution. For categorical variables, we used either the χ2 or the Fisher’s exact test to assess associations and linear-by-linear association for assessing trends. For continuous variables, we used either the Student's t test or the Mann–Whitney U test for associations and analysis of variance or the Kruskal–Wallis test for comparisons between three groups or more. Post hoc comparisons were applied with Bonferroni or Dunnet T3 corrections of significance level as appropriate.

Multivariate logistic regression modeling (forward stepwise method) was used to identify variables associated with receiving two or more compassionate/experimental drugs (vs. 0–1). Variable inclusion criteria included fulfillment of logical clinical associations as well as statistical significance in univariate analysis (p < 0.05). Variables were retained when their inclusion provided the most statistically significant improvement of model fit in repeated model iterations. To study the adjusted association between receiving two or more compassionate/experimental drugs (vs. 0–1) and all-cause survival and in-hospital LOS, we used multivariate logistic regression modeling (Enter method) and analysis of covariance. All statistical tests were two-tailed and a P value of 5% or less was considered statistically significant. Analyses were performed using IBM SPSS™ (v.25).

Results

Participants

Figure 1 illustrates the study inclusion/exclusion process. Overall, 319 patients were admitted to SZMC during the study period and among these 292 were included.

Fig. 1
figure 1

Flow chart for study inclusion and exclusion

Full size image

Description of the Study Cohort as a Whole

Patient characteristics are presented in Supplement Tables 1a and 1b. The mean age of the included patients was 60.18 ± 18.96 years, and 59.2% (n = 173/292) of them were male. Most (68%, n = 198/292) had at least one prior medical condition. The most common comorbidity was essential hypertension. The survival rate in this early cohort of patients admitted due to SARS-CoV-2 related symptoms was 90% (n = 263/292).

Critically Ill Versus Non Critically Ill Patients

The characteristics of the patients defined as critically ill versus those described as non-critically ill for the purpose of this study are presented in Supplemental Tables 2a and 2b. Critically ill patients were older, had more background diseases (i.e., hypertension, diabetes mellitus, and ischemic heart disease, and had a higher BMI).Critically ill patients also differed from non-critically ill patients in most vital signs and laboratory parameters (Supplemental Table 2b). The most striking differences between the groups were the values of oxygen saturation, which were lower among critically ill patients (85.3% ± 9.4 vs. 93.2% ± 3.9, P < 0.001) and in CRP levels which were higher among critically ill patients (16.6 ± 12 vs. 6 ± 6, P < 0.001).

In order to verify that maximal respiratory support can also be used as a surrogate for patient severity, we also studied mortality rates and LOS versus maximal respiratory support. Figure 2a shows mortality in relation to maximal respiratory support. Survival decreased with escalating support; the overall comparison between the groups was statistically significant (P < 0.001) and the trend of increasing mortality with increasing maximal respiratory support was also significant (P < 0.001, linear by linear association). Figure 2b shows LOS in relation to maximal respiratory support. LOS also increased in relation to escalating maximal respiratory support (P < 0.001, Kruskal–Wallis).

Fig. 2
figure 2

a Rate of survival in relation to the degree of respiratory support. b Length of hospital stay (days) in relation to the degree of respiratory support

Full size image

Number of Compassionate\Experimental Drugs Administered to COVID-19 Patients and Relation to Patient Severity

The compassionate\experimental drugs administered to patients during the study period were azithromycin, hydroxychloroquine, corticosteroids, remdesivir, convalescent plasma, lopinavir, opaganib, and tocilizumab. Supplemental Fig. 1 shows the distribution of the number of compassionate\experimental drugs administered to study patients. Figure 3 shows the increasing proportion of patients who received two compassionate\experimental drugs or more as respiratory support escalated (P < 0.001, linear by linear association). The number of drugs administered to critically ill patients was significantly higher than the number of drugs administered to patients not critically ill (0.55 ± 0.689 vs. 1.92 ± 1.082, P < 0.001). Two or more compassionate\experimental drugs were administered to 65.4% (34/52) of critically ill patients versus 11.3% (27/240) of non-critically ill patients (P < 0.001).

Fig. 3
figure 3

Percentage of patients receiving 0–1 or 2 + experimental/compassionate drugs in relation to the degree of respiratory support

Full size image

Association Between Patient Characteristics and the Likelihood of Receiving More Than Two Experimental/Compassionate Drugs (Unadjusted and Adjusted)

In univariate analysis, male sex, the presence of DM, smoking and a higher BMI, higher temperatures and heart rates and lower O2Sat as well as elevated hematocrit, neutrophil count, CRP, and LDH at the time of hospitalization were significantly associated with receiving more than two experimental/compassionate drugs (Supplement Table 3). In the multivariate model adjusting for all of these variables only two variables remained significant: higher oxygen saturation decreased the likelihood of receiving more than two experimental/compassionate drugs during hospitalization (OR 0.747, 95% CI 0.657–0.85, P < 0.001) whereas higher CRP increased the likelihood of receiving more than two experimental/compassionate drugs during hospitalization (OR 1.16, 95% CI 1.083–1.243, P < 0.001) (Supplement Table 3).

Association Between Number of Experimental/Compassionate Medications Received and Outcomes

Survival to hospital discharge was unrelated to the incremental (ordinal) number of experimental/compassionate medicationsadministered (Fig. 4). Survival was also similar among patients who received 0–1 drugs and among those who received 2 drugs or more [90.5%, (n = 210/232) vs. 86.7% (n = 52/60), P = 0.328]. Multivariate analysis adjusting for patient severity as reflected by CRP and O2Sat also showed no relation between the number of experimental/compassionate medications received (0–1 vs. 2 or more) and survival to hospital discharge (p = 0.99) (Supplement Table 4a).

Fig. 4
figure 4

Rate of survival by number of experimental/compassionate drugs received

Full size image

LOS was higher among patients receiving more experimental/compassionate medications (Supplemental Fig. 2) (P < 0.001). Figure 5 illustrates the differences in LOS between patients who received 0–1 drugs (7.85 ± 6.9, median 6, IQR 4–10) and those who received 2 or more drugs (18.28 ± 14.76, median 13, IQR 8–26.75 days) (P < 0.001).

Fig. 5
figure 5

Hospital length of stay in patients receiving 0–1 and 2 or more experimental/compassionate drugs

Full size image

Association Between Number of Experimental/Compassionate Medications Received and LOS (Adjusted Analysis)

Even after adjusting for CRP and oxygen saturation, LOS remained significantly longer in patients receiving 2 drugs or more when compared to patients receiving 0–1 experimental/compassionate drugs (P < 0.001, analysis of covariance) (Supplement Table 4b).

Discussion

This observational study of 292 patients with COVID-19 admitted during the first pandemic wave shows that almost a third of COVID-19 patients (31.5%) received one experimental drug during their hospitalization, and that one in five patients (20.9%) received more than 1 experimental drug. We also found a clear correlation between patient severity as assessed by laboratory parameters, as has been previously reported [9], vital signs, the level of respiratory support administered, and the number of experimental/compassionate drugs administered to these patients. We found no association between the number of drugs administered and mortality (P = 0.33) and this lack of association persisted even after adjustment for patient severity as reflected by CRP and O2Sat at presentation (P = 0.99). Conversely, we did find an association between the number of drugs administered and patient LOS (0.001), and this association persisted after adjustment for the same parameters of patient severity. Put together, these findings suggest that patients who were in worse condition upon arrival received more respiratory support and drugs during admission, and also had longer LOS but still survived less well despite these efforts.

Studies that attempt to adjust for multiple experimental drug administration may have a problem isolating the effect of a single drug. For example, the dexamethasone trial from the RECOVERY group was designed to have a similar rate of co-treatments in the intervention and non-intervention arms. This study eventually concluded that steroids may be helpful only in the sub-category of patients requiring oxygen support and in those requiring invasive ventilation. Although the two study arms for the main outcome of the full cohort had equal co-treatment distributions, this information was not provided for the subgroup analyses [3].

Similarly, although RECOVERY reported a mortality benefit for tocilizumab, the study did not identify a particular subgroup of hospitalized patients on conventional oxygen therapy who benefited most from receiving the drug [3]. In a systemic review and meta-analysis by Hariyanto et al. [10], tocilizumab was found to reduce the mortality rate in severely ill patients, requiring oxygen support, with high interleukin-6 levels; however, it did not alter the severity of disease and LOS.

Our data suggest an association between patient severity and polypharmacy which may have biased the results.

Finally, prescription of more than one experimental drug may directly alter the effect of another drug. For example, green tea, which has compounds inducing death of cancer cells in multiple myeloma, also prevents proteasome inhibitor-induced cell death in this disease [11, 12]. Additional examples are with vitamin C and proteasome inhibitors, and curcumin and cyclophosphamide [13, 14].

Similar to other studies, we also noted that patients defined as critically ill were older, had higher BMI, and had more background diseases (hypertension, diabetes mellitus, ischemic heart disease) than those who were less sick [15,16,, 16]. In this early cohort, we had a high survival rate (90%). Similarly, high survival rates have been described in other cohorts: Domingo et al. [17] described an 87% survival rate in the first pandemic wave among hospitalized patients.

This study is the first to examine the relationship between patient severity and multiple drug administration. This crucial confounder has not been discussed with regards to the outcomes described in subgroup analyses of all the major studies regarding COVID-19. Our data were collected in real time by a researcher dedicated to the topic (O.A.) which contributed to data reliability. Our findings highlight an issue that must be methodologically addressed in future studies.

Our study has several limitations. The practice observed in this single-center study may not be generalizable elsewhere. However, prior multi-center studies have shown a 35.6% rate of at least one more concomitant drug administration [4]. The circumstances of the first pandemic wave resulted in a relatively high rate of missing data with regards to several parameters (e.g., fibrinogen, smoking). We did not include these parameters in the analysis, but we do show the extent of missing data. The main difference between patients with missing data was departmental allocation. We could not study association of placement with survival as patients were often transferred from one location to another. Like most pragmatic studies, we did not interfere in clinician treatment decisions. Although the data were collected in real time, the analysis was conducted retrospectively. Also, we selected the parameters to be collected for this study based on our experience in real time. These may not reflect the full spectrum of data available to physicians during the decision-making process in later pandemic waves. Our dataset is too small to elicit the particular effect of any single drug, to study the spectrum of possible complications caused by the medications administered, or to study time effects. Finally, our study was not powered to examine the association between treatment and outcomes. The models we designed to adjust this association for patient severity may therefore also be overfitted. Nonetheless, the variables selected for adjusting for patient severity were stable across multiple iterations as were our findings.

At the beginning of the COVID-19 pandemic, vaccines were unavailable and multiple compassionate and experimental drugs were used in a desperate effort to find a cure for the disease. While attempts to find a cure should be praised, when compassionate and experimental drugs are administered, measures should be taken to ensure that not only the individual patient benefits but others do well. This study highlights the issue of polypharmacy and its relationship to patient severity. Treatment with multiple drugs may obscure the effect of a specific drug [7]. The positive effect of potential treatments may have been missed, due not only to a Simpsons paradox (negative confounding) but also due to severity bias. Our study reflects practice at the time of greatest uncertainty. If our findings have not persisted across the following pandemic waves, there is still hope of identifying a weak treatment signal.