The emergence of SARS-CoV-2 variants is jeopardizing the effectiveness of current vaccines and limiting the application of monoclonal antibody-based therapy for COVID-19 (refs. ^1,2). Here we analysed the memory B cells of five naive and five convalescent people vaccinated with the BNT162b2 mRNA vaccine to investigate the nature of the B cell and antibody response at the single-cell level. Almost 6,000 cells were sorted, over 3,000 cells produced monoclonal antibodies against the spike protein and more than 400 cells neutralized the original SARS-CoV-2 virus first identified in Wuhan, China. The B.1.351 (Beta) and B.1.1.248 (Gamma) variants escaped almost 70% of these antibodies, while a much smaller portion was impacted by the B.1.1.7 (Alpha) and B.1.617.2 (Delta) variants. The overall loss of neutralization was always significantly higher in the antibodies from naive people. In part, this was due to the IGHV2-5;IGHJ4-1 germline, which was found only in people who were convalescent and generated potent and broadly neutralizing antibodies. Our data suggest that people who are seropositive following infection or primary vaccination will produce antibodies with increased potency and breadth and will be able to better control emerging SARS-CoV-2 variants.