During the coronavirus disease-2019 (COVID-19) pandemic, severe acute respiratory syndrome-related coronavirus-2 (SARS-CoV-2) has led to the infection of millions of people and has claimed hundreds of thousands of lives. The entry of the virus into cells depends on the receptor-binding domain (RBD) of the spike (S) protein of SARS-CoV-2. Although there is currently no vaccine, it is likely that antibodies will be essential for protection. However, little is known about the human antibody response to SARS-CoV-21–5. Here we report on 149 COVID-19-convalescent individuals. Plasma samples collected an average of 39 days after the onset of symptoms had variable half-maximal pseudovirus neutralizing titres; titres were less than 50 in 33% of samples, below 1,000 in 79% of samples and only 1% of samples had titres above 5,000. Antibody sequencing revealed the expansion of clones of RBD-specific memory B cells that expressed closely related antibodies in different individuals. Despite low plasma titres, antibodies to three distinct epitopes on the RBD neutralized the virus with half-maximal inhibitory concentrations (IC50 values) as low as 2 ng ml−1. In conclusion, most convalescent plasma samples obtained from individuals who recover from COVID-19 do not contain high levels of neutralizing activity. Nevertheless, rare but recurring RBD-specific antibodies with potent antiviral activity were found in all individuals tested, suggesting that a vaccine designed to elicit such antibodies could be broadly effective. Although rare, antibodies against the receptor-binding domain of SARS-CoV-2 that showed potent antiviral activity were obtained from all tested convalescent individuals, suggesting that a vaccine designed to elicit such antibodies could be broadly effective.