Accurate assays for the detection of antibodies to SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) are essential for the control of the COVID-19 (coronavirus disease 2019) pandemic. Here, we report antibody and antibody-avidity assays, relying on near-infrared-fluorescence amplification by nanostructured plasmonic gold substrates, for the simultaneous detection of antibodies to the S1 subunit of the spike protein and to the receptor binding domain of SARS-CoV-2 in human serum and saliva, and for quantifying immunoglobulin avidities against coronavirus antigens from SARS-CoV-2, SARS-CoV-1 and the common-cold viruses OC43, HKU1, NL63 and 229E. The antibody assay detected immunoglobulin M in 87% (52 of 60) COVID-19-positive serum samples collected 6 or more days after symptom onset (and the immunoglobulins M and G in all 33 samples collected at least 15 days after symptom onset), and correctly classified 456 out of the 457 COVID-19-negative serum samples tested (424 of them collected before the pandemic, including 73 that were positive for other viruses). We used the antibody-avidity assay to study antibody-maturation patterns, anamnestic responses, and cross-immunity to the common-cold coronaviruses.