Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a prototypical sarbecovirus, causes coronavirus disease 2019 (COVID-19) and is associated with significant morbidity and mortality [1, 2]. SARS-CoV-2 neutralizing antibodies (NtAb) presumably play a pivotal role in preventing infection and may promote virus clearance [3, 4]. In support of these assumptions, passive transfer of two mAbs blocking SARS-CoV-2 interaction with angiotensin-converting enzyme II receptor as monotherapy protected rhesus macaques from infection [5]. Moreover, transfusion of plasma from immune individuals with high NtAb titers seemed to be associated with improved clinical outcomes in critically ill COVID-19 patients [6, 7]. Virus neutralization assays, either using live native SARS-CoV-2 virus, engineered SARS-CoV-2 pseudotyped viruses, or replication-competent SARS-CoV-2 chimeric viruses are cumbersome, require specialized facilities, and are time consuming [8,9,10]. A large number of enzyme-linked (ELISA) or chemiluminescent immunoassays (CLIA) detecting antibodies that bind SARS-CoV-2 structural proteins have been commercialized [11]. Whether antibody levels measured by these serological assays can be used as a proxy for serum neutralizing activity is a relevant issue, which has been dealt with in several peer-reviewed or preprint studies [12,13,14,15,16,17,18,19,20], but demands further investigations. Here, we focused on evaluating the degree of correlation between NtAb binding the SARS-CoV-2 S protein, which is known to elicit the most potent antibodies for virus neutralization [21, 22] and SARS-CoV-2 IgG levels measured by four commercially available semiquantitative immunoassays targeting the S protein in sera drawn from hospitalized COVID-19 patients.

Material and methods

Serum specimens and patients

In this retrospective study, a total of 90 sera from 51 non-consecutive patients with laboratory-confirmed SARS-CoV-2 infection by RT-PCR that were admitted to Hospital Clínico Universitario of Valencia between March 5 and April 30, 2020, were included [23, 24]. The only patients’ inclusion criterium was the availability of leftover sera obtained for routine SARS-CoV-2 serological testing. Sera had been cryopreserved (− 20 °C) for a maximum of one month since collection, and never thawed prior to performing the analyses reported herein. All serological assays were performed within one week as described below. The demographic, clinical, and laboratory data of these patients are displayed in Table 1. Forty-one sera were obtained within the first two weeks (< day 15) after the onset of symptoms, at a median of 11 days (range, 5–14 days), and 49 afterward (≥ 15 days, at a median of 23 days; range, 15–41 days). Sequential specimens were available from 20 out of the 51 patients (median 3 specimens per patient; range 2 to 6). A total of 20 pre-pandemic sera from healthy individuals collected within 2019, of which 10 belonged to patients with prior endemic coronavirus infections (HCoV-229E, n = 8; HCoV NL63, n = 1; HCoVHKU, n = 1), were included as controls. This study was approved by the Research Ethics Committee of Hospital Clínico Universitario INCLIVA (March 2020).

Table 1 Demographic, clinical, and laboratory characteristics of patients with COVID-19
Full size table

SARS-CoV-2 neutralizing antibody assay

A green fluorescent protein (GFP) reporter-based pseudotyped virus neutralization assay using a non-replicative vesicular stomatitis virus (VSV lacking the G protein) backbone coated with the SARS-CoV-2 spike (S) protein was used for neutralization assays on Vero cells, as previously described [24]. Briefly, sera were heat-inactivated for 30 min at 56 °C then brought to an initial 10-fold dilution, followed by four 4-fold dilutions in duplicate. Each dilution was mixed with an equal volume containing 1250 plaque-forming units of the VSV-S virus and incubated at 37 °C for 1 h. Subsequently, the mixture was added to Vero cells and incubated for 18 h, after which GFP expression was measured using a live cell microscope system (IncuCyteS3; Sartorius). All sera which did not reduce viral replication by 50% at 1/20 dilution were considered non-neutralizing and were arbitrarily assigned a value of 1/10. All sera that did not result in > 70% recovery of GFP signal at the highest antibody dilution were retested using 5-fold dilutions ranging between 100- and 12,500-fold. Finally, the antibody dilution resulting in 50% virus neutralization (NtAb50) was calculated using the drc package (version 3.0-1) in R via a 2 parameter logistic regression model (LL.2 model). In a previous study, by testing 90 sera from.

Commercial SARS-CoV-2 IgG immunoassays

Four commercially available semiquantitative immunoassays were used in the current study. Performance and interpretation of results were done in accordance with the respective manufacturer’s instructions. The LIAISON SARS-CoV-2 S1/S2 IgG chemiluminescent assay (DiaSorin S.p.A., Saluggia, Italy) detects IgG antibodies directed against a recombinant S protein (S1/S2). Samples displaying < 12.0 AU/mL are considered negative, those ranging between 12.0 and 15.0 AU/mL are undetermined, and those > 15 AU/mL are deemed as positive. The Euroimmun SARS-CoV-2 IgG ELISA (Euroimmun, Lübeck, Germany) uses a recombinant S1 domain of the S protein as a target. Results are expressed as a ratio, calculated by dividing the optical densities of the sample by those of an internal calibrator provided with the test kit. The cut-off index (COI) for samples to be considered positive was ≥ 1.1 and inconclusive from 0.8 and 1.09. The MAGLUMI 2019-nCoV IgG is an indirect CLIA for assessment of IgG antibodies against SARS-CoV-2 S and nucleocapsid (N) proteins on the fully automated MAGLUMI analyzers (SNIBE—Shenzhen New Industries Biomedical Engineering Co., Ltd., Shenzhen, China). A test result ≥ 1.10 AU/mL is considered positive. The COVID-19 ELISA IgG (Vircell Spain S.L.U., Granada, Spain) is an enzyme immunoassay that detects IgGs targeting the S1 and N proteins. Sera displaying antibody indices (AI) < 1.4 are considered negative, those between 1.4 < 1.6 inconclusive, and those > 1.6 positive. For all the assays, inconclusive results were considered as positives for analysis purposes.

Definitions

Here, NtAb titers ≥ 1/160 were considered high as this is the minimum NtAb50 titer of plasma from COVID-19 convalescent individuals recommended by the FDA for therapeutic use [25]. Trajectories of antibody titers/levels were categorized as ascendant, descendant, fluctuating, or constant. To this end, variations of antibody levels > 10% across sequential specimens were deemed to be relevant, as the intra-run coefficient of variation was below this figure for all the immunoassays (all samples from a given individual were assayed in the same run).

Statistical methods

Test performances were evaluated for their sensitivity with the associated 95% confidence interval (CI). Cohen’s kappa (κ) statistic was used to evaluate the qualitative agreement between immunoassays. Spearman’s rank test was used to assess the correlation between continuous variables using the entire dataset. To identify the optimal SARS-CoV-2 IgG levels measured by commercial immunoassays predicting NtAb50 titers ≥ 1/160, receiver operating characteristic (ROC) curve analysis was performed. Two-sided exact P values are reported, and a P value < 0.05 is considered statistically significant. The analyses were performed using SPSS version 20.0 (SPSS, Chicago, IL, USA). Kinetics of antibody titers/levels were categorized as ascendant (an increase of antibody levels > 10% compared to previous sampling point), descendant (any decrease), fluctuating, or constant.

Results

Patients characteristics

All 51 patients presented with pneumonia and imaging or laboratory findings compatible with COVID-19 and were hospitalized in either the pneumology ward (n = 27) or the intensive care unit (ICU; n = 24). As shown in Table 1, most patients (69%) had one or more comorbidities and displayed high serum levels of several pro-inflammatory biomarkers at the time of serological testing. Four ICU patients eventually died.

Specificity of NtAb and SARS-CoV-2 IgG immunoassays

Out of the 20 control sera, none returned positive results by any of the immunoassays used in the study. Thus, the specificity of CLIA and the NtAb assays was 100% (95% CI, 83.9–100%).

Agreement between NtAb and SARS-CoV-2 IgG immunoassays results

Qualitative results returned by immunoassays were evaluated either considering the entire dataset or grouping sera according to the time of sampling after the onset of symptoms (< 15 days or ≥ 15 days) (Table 2). As shown in Table 3, overall, results provided by the COVID-19 ELISA IgG test best matched those obtained with the NtAb assay (κ, 0.84; 95% CI, 0.63–1), followed by those of the Euroimmun SARS-CoV-2 IgG ELISA (κ, 0.52; 0.52; 95% CI, 0.22–0.81), LIAISON SARS-CoV-2 S1/S2 IgG (κ, 0.5; 95% CI, 0.2–0.78), and MAGLUMI 2019-nCoV IgG (0.4; 95% CI, 0.2–0.77). The same trend was observed when sera collected either < 15 days or ≥ 15 days after the onset of symptoms were analyzed separately. Notably, the concordance between results returned by the NtAb assay and the COVID-19 ELISA IgG was 100% for sera obtained at ≥ 15 days following symptom onset.

Table 2 Performance of an antibody neutralization method using a reporter-based pseudotyped virus (vesicular stomatitis virus pseudotyped with the SARS-CoV-2 spike protein) and four commercial SARS-CoV-2 IgG immunoassays for the diagnosis of COVID-19
Full size table
Table 3 Agreement between the results of a reporter-based pseudotyped virus antibody neutralization method (vesicular stomatitis virus pseudotyped with the SARS-CoV-2 spike protein) and four commercial SARS-CoV-2 IgG immunoassays
Full size table

The sensitivity of NtAb and SARS-CoV-2 IgG immunoassays

Overall, the most sensitive tests were the GFP reporter-based pseudotyped virus neutralization assay and the COVID-19 ELISA IgG, followed by the MAGLUMI 2019-nCoV IgG, the Euroimmun SARS-CoV-2 IgG ELISA, and the LIAISON SARS-CoV-2 S1/S2 IgG (Table 4). Differences in sensitivity were more apparent when sera were collected early after the onset of symptoms (< 15 days) were analyzed independently, and these tended to decrease in sera obtained at a later time point (Table 4).

Table 4 Clinical sensitivity of an antibody neutralization method using a reporter-based pseudotyped virus (vesicular stomatitis virus pseudotyped with the SARS-CoV-2 spike protein) and four commercial SARS-CoV-2 IgG immunoassays for the diagnosis of COVID-19
Full size table

Correlation between NtAb50 titers and IgG levels measured by SARS-CoV-2 IgG immunoassays

NtAb50 titers ranged from ≤ 1/20 (undetectable) to 1/12,500. As shown in Fig. 1, overall, the correlation between NtAb50 titers and SARS-CoV-2 IgG levels, as returned by the corresponding immunoassay, was strong for the Euroimmun SARS-CoV-2 IgG ELISA (rho = 0.73) and moderate (rho = 0.48 to 0.59) for the remaining platforms. In all instances, correlations were positive and statistically significant. The degree of correlation between NtAb50 titers and SARS-CoV-2 IgG levels was substantially better for sera collected within the first two weeks after the onset of symptoms (rho = 0.54–0.80) than for those obtained afterward (rho = 0.30–0.52), irrespective of the commercial immunoassay employed.

Fig. 1
figure 1

Correlation between NtAb50 titers measured by a SARS-CoV-2-S pseudotyped virus neutralization assay and IgG levels measured by commercial SARS-CoV-2 IgG immunoassays in all sera (a), in sera obtained within the first two weeks after the onset of symptoms (<15 days; b), and in sera collected afterward (≥ 15 days; c). For (a), Rho and P values were as follows: Euroimmun SARS-CoV-2 IgG , 0.73 and P<0.001; MAGLUMI 2019-nCoV IgG, 0.48 and P<0.001; LIAISON SARS-CoV-2 S1/S2 IgG, 0.52 and P<0.001. COVID-19 ELISA IgG, 054 and P<0.001. For (b) Rho and P values were as follows: 0.80 and P<0.001; 0.54 and P<0.001; 0.68 and P<0.001; 0.63 and P<0.001, respectively. For (c) Rho and P values were as follows: 0.55 and P<0.001; 0.30 and P<0.03; 0.32 and P=0.02; 0.39 and P=0.005, respectively

Full size image

Inference of high NtAb50 titers by using SARS-CoV-2 IgG immunoassays

Seventy-four out of 90 sera displayed high NtAb50 titers (≥ 1/160). ROC analysis was performed to determine the IgG levels measured by commercial immunoassays that predict NtAb50 titers of such a magnitude (Fig. 2). Specificity was prioritized at the expense of sensitivity for threshold selection. The data are shown in Table 5. The best combination of specificity, sensitivity, and predictive values was achieved by the LIAISON SARS-CoV-2 S1/S2 IgG threshold, with minimal differences across the remaining platforms.

Fig. 2
figure 2

ROC curve analysis for establishing the optimal SARS-CoV-2 IgG threshold levels that predict the presence of high NtAb50 titers (≥ 1/160) in hospitalized patients with COVID-19 for the Euroimmun SARS-CoV-2 IgG ELISA (a), LIAISON SARS-CoV-2 S1/S2 IgG assay (b), MAGLUMI 2019-nCoV IgG (c), and COVID-19 ELISA IgG (d) assays

Full size image
Table 5 SARS-CoV-2-S-IgG threshold levels measured by four commercial immunoassays for predicting neutralizing antibody titers ≥ 1/160 measured by a reporter-based pseudotyped virus antibody neutralization method (vesicular stomatitis virus pseudotyped with the SARS-CoV-2 spike protein)
Full size table

Longitudinal follow-up of NtAb50 titers and SARS-CoV-2 IgG levels measured by immunoassays

We selected patients (n = 11) with 3 or more sera available for kinetic studies. Kinetics of NtAb50 titers and SARS-CoV-2 IgG levels determined by commercial immunoassays varied notably across patients and were dependent on the immunoassay considered. Concordance between trajectories of NtAb50 titers and SARS-CoV-2 IgG levels was highest when using the COVID-19 ELISA IgG (6 out of 11 patients) assay, followed by MAGLUMI 2019-nCoV IgG (5 out of 11), and LIAISON SARS-CoV-2 S1/S2 IgG and Euroimmun IgG (both 3 out of 11) assays. Representative cases of concordant and discordant trajectories are depicted in Fig. 3.

Fig. 3
figure 3

Kinetics patterns of serum NtAb50 titers quantitated by a SARS-CoV-2-S pseudotyped virus neutralization assay and SARS-CoV-2 IgG levels measured by commercial immunoassays in two representative patients displaying concordant (upper panels) or discordant (lower panels) kinetics

Full size image

Discussion

A major role in preventing SARS-CoV-2 infection and blunting virus spread in COVID-19 patients is predicted for NtAb [3,4,5]. Quantitation of serum NtAb titers in plasma from convalescent COVID-19 patients is recommended for optimal selection of specimens for passive transfer therapies; the FDA recommends the use of plasma with NtAb50 levels ≥ 1/160 [22]. NtAb assays, either using SARS-CoV-2 strains, pseudotyped viruses, or chimeric viruses are technically demanding, require specialized facilities (biosafety level 3 for live SARS-CoV-2 or level 2 for the pseudotyped viruses), and cannot be implemented in routine practice. Thus, it is of interest to find alternative methods to obtain reliable information regarding the neutralizing activity of sera that are simple, fast, high-throughput assays, and commercially available. Here, we compared the performance of a GFP reporter-based pseudotyped virus neutralization assay (VSV-SARS-CoV-2 S) and four commercial immunoassays also targeting the SARS-CoV-2 S protein, which in our experience displayed a specificity of 100%, for the diagnosis of COVID-19 and assessed the extent to which NtAb50 titers and SARS-CoV-2 IgG levels measured by these assays correlate. In our experience, all immunoassays displayed a specificity of 100%. We chose to include two assays targeting the SARS-CoV-2 N protein in addition to the S protein (MAGLUMI 2019-nCoV IgG and the COVID-19 ELISA IgG) to determine whether the combined use of both antigens would impact on the degree of correlation between SARS-CoV-2 IgG levels and NtAb50 titers measured by a NtAb assay targeting exclusively the S protein. Prior studies assessing this relationship have been published for the Euroimmun SARS-CoV-2 IgG ELISA and LIAISON SARS-CoV-2 S1/S2 IgG [12, 13, 15, 18,19,20], but not, to the best of our knowledge, for the MAGLUMI 2019-nCoV IgG and COVID-19 ELISA IgG assays. It must be highlighted that antibody titers measured by the NtAb assay used in the current study strongly correlate with those quantitated by assays using wild-type SARS-CoV-2 [26].

Here, the overall agreement between qualitative (positive/negative) results returned by the NtAb assay and the four commercially available immunoassays was fairly good, ranging from 97% for the COVID-19 ELISA IgG assay to 88% for LIAISON SARS-CoV-2 S1/S2 IgG and MAGLUMI 2019-nCoV IgG assays. Concordance across results was better for sera obtained at late times after the onset of symptoms (≥ 15 days), likely reflecting the existence of differences in the kinetics of NtAb and SARS-CoV-2 IgG levels measured by commercial immunoassays early after infection. In line with this finding, the COVID-19 ELISA IgG performed comparably to the NtAb assay in terms of overall sensitivity (92.2%), ranking first across the evaluated commercial immunoassays. As previously reported, the sensitivity of all immunoassays increased after the second week following the onset of symptoms [12, 13, 15, 16, 18,19,20]. The overall degree of correlation between NtAb50 antibody titers and SARS-CoV-2 IgG levels was strong for the Euroimmun IgG assay but low to moderate for the other commercial immunoassays. The fact that the Euroimmun IgG assay detects antibodies against S1, which contains the RBD domain and concentrates immunodominant epitopes eliciting high-affinity NtAb antibodies [3, 5, 21, 22], may account for this finding. IgG antibodies lacking neutralizing activity and recognizing epitopes within S2 and N proteins (additional targets in the remaining immunoassays) may compete with those binding S1, thus decreasing the degree of correlation. Interestingly, in all cases, correlations were better for sera collected early after the onset of symptoms than for those obtained at later times; in line with this, NtAb trajectory could not be inferred in a large fraction of patients by sequentially monitoring SARS-CoV-2 IgG levels, regardless of the commercial immunoassay used. However, threshold levels of IgGs predicting high NtAb50 titers (≥ 1/160) with high specificity (93%) could be established for all immunoassays, although sensitivities were overall poor.

In the absence of a reference panel of sera with standard levels of NtAb antibodies, comparison across studies addressing the above issue is not straightforward, as the biological characteristics of the neutralization assay, the timing of sera collection, and the clinical severity of COVID-19 patients differ notably between them and may explain certain discrepancies. Namely, Weidner et al. [12] reported a strong correlation between IgG levels measured by the LIAISON SARS-CoV-2 S1/S2 IgG and NtAb50 titers quantified by a wild-type SARS-CoV-2 assay (rho = 0.75 versus rho = 0.59 in the current study) in sera from convalescent COVID-19 patients. The degree of correlation reported by Muecksh et al. [20] in a comparable population using an HIV pseudotyped platform was even higher (rho = 0.82). Likewise, GeurtsvanKessel et al. [13] found a strong correlation (rho = 0.75) using a live SARS-CoV-2 microneutralization assay and sera from a mixed patient population in terms of COVID-19 severity. Despite the above-highlighted differences across studies, our correlation data agree with those of Weidner et al. [12] and GeurtsvanKessel et al. [13] for the Euroimmun IgG assay (rho 0.75 and 0.76, respectively, versus 0.73 herein) and those of Criscuolo et al. [15] who reported a low correlation between IgG levels measured by the LIAISON SARS-CoV-2 S1/S2 IgG and NtAb50 titers quantified by using a live SARS-CoV-2 platform in sera obtained within 15 days after the onset of symptoms in a small cohort of COVID-19 hospitalized patients. Moreover, Bonelli et al. [18] found that the probability of having NtAb50 titers > 1/160, as determined by a SARS-CoV-2 wild-type assay, was 92% when LIAISON SARS-CoV-2 S1/S2 IgG levels were > 80 AU/mL (90 AU/mL in the current study for a positive predictive value of 98%). Also, in line with a previous report [20], the trajectory of NtAb50 titers could not be predicted by assessing SARS-CoV-2 IgG levels when measured by commercial immunoassays.

The current study has several limitations that deserve comment. First, due to small sample size, further studies are warranted to confirm our findings. Second, our conclusions strictly apply to COVID-19 hospitalized patients. Whether or not these can be extrapolated to asymptomatic or mildly symptomatic individuals not requiring hospitalization needs to be proven.

In summary, our data indicate that the reliability SARS-CoV-2 IgG commercial immunoassays targeting the S protein to infer neutralizing activity against this protein in sera from hospitalized COVID-19 patients varies widely across tests, this being better for assays targeting exclusively the S protein than for those detecting antibodies to the S and N proteins in combination, and depends upon the time of sera collection after the onset of symptoms.