Real-world experience of SARS-CoV-2 antibody assays in UK healthcare workers

Summary

What is known?

There are a variety of different antibody tests for the detection of previous SARS-CoV-2 infection, which were widely used to determine seroprevalence among healthcare workers and the general population. Concerns were raised as these antibody assays were not validated in such populations and may not produce meaningful results.

What is the question?

What is the seroprevalence of SARS-CoV-2 antibodies among two cohorts of healthcare workers and how have two different antibody assays, Abbott and Roche, performed within these cohorts?

What was found?

The seroprevalence of two similar UK regions was 7.9% using the Abbott and 13% using the Roche assay. The real-world sensitivities were only 39.8% and 81% respectively. Male sex, South Asian ethnicity and healthcare assistants were most strongly associated with antibody positivity.

What is the implication for practice now?

Real-world data suggests antibody assays do not perform to manufacturers’ specifications when used on healthcare workers. Side-by-side analysis of different assays within their target populations may be necessary to ensure valid and meaningful results.

Introduction

The SARS-CoV-2 global pandemic has demanded rapid mobilisation of healthcare resources and unprecedented public health interventions. The UK has now surpassed three million cases and 90,000 deaths.¹

Antibody testing for SARS-CoV-2 has been a major component of public health campaigns worldwide.^2,3 However, concerns have been raised about the rapid uptake of testing, questioning the relevance and also the validity of tests available within the target populations.^4,5 Further, the interpretation of results is limited as the presence of antibodies does not guarantee immunity to future infection.⁶

In the UK, serum antibody testing was introduced and offered to almost all NHS staff from 25 May 2020.⁷ Global disparities in seroprevalence among healthcare workers have been seen^8–13 and attributed to local incidence, personal protective equipment (PPE) availability and hospital organisation.^14,15 Published and in-press data highlight significant disparities in seroprevalence among differing ages, ethnicities, occupations, workplaces and geographical areas, with some of the highest rates recorded in the UK.^8,12,15 However, differences in assay may also be a contributor and this has yet to be appreciated in a real-world dataset.

Western Sussex Hospitals NHS Foundation Trust (WSHT) and Brighton and Sussex University Hospitals (BSUH) worked collaboratively under the same management during the first wave of the pandemic. Epidemiological differences across the region are represented in Fig 1 (data from Public Health England [PHE]¹). It can be appreciated that during the first wave of the pandemic, infection rates were higher in WSHT regions than BSUH. The estimated general seroprevalence of the South East is 4.7%.¹⁶

• Download figure
• Open in new tab
• Download powerpoint

Fig 1.

Epidemiological differences across the region served by Western Sussex Hospitals NHS Foundation Trust (WSHT) and Brighton and Sussex University Hospitals (BSUH). Data from Public Health England.¹

PHE have now evaluated eight different antibody assays, the first of which were provided by Abbott (for which PHE report a sensitivity of 92.7% and specificity of 100%¹⁷) and Roche (sensitivity of 83.9% and specificity of 100%¹⁸). In line with centralised allocation of testing platforms and local availability of analysers, WSHT used the Abbott assay and BSUH used the Roche assay for staff antibody testing.

The aim of this study was to determine the seroprevalence of SARS-CoV-2 antibodies among healthcare workers in these regions and compare the performance of the different antibody assays within their respective cohorts.

Methods

Results

Overall, 26,861 SARS-CoV-2 antibody tests were processed across both sites over a 4-month period: 12,388 at WSHT and 14,473 at BSUH. Positive assays were present in 978 (7.9%) WSHT staff and 1,880 (13%) at BSUH. Positive PCR results were recorded in 566 staff members at WSHT and 163 staff members at BSUH prior to antibody testing, giving an overall known infection rate of 4.9% and 1.1% at each site respectively.

Antibody tests occurred a median 97 days after symptoms onset in those with a positive PCR in the WSHT cohort, and 53 days after PCR and 61 days after symptom onset in the BSUH cohort. Symptom reporting was marginally higher at BSUH than WSHT (29.6% versus 27.6% respectively). The rates of antibody detection in asymptomatic staff were 3.3% for WSHT and 6.9% for BSUH (Table 1). When comparing the antibody results to a prior positive PCR result (ie the current gold standard test for infection with SARS-CoV-2), only 39.8% of staff at WSHT and 81% of staff at BSUH demonstrated a positive antibody assay. In staff at WSHT who both experienced symptoms and demonstrated a positive PCR result (n=362), only 54.7% demonstrated antibodies. At BSUH, only two cases with symptoms and a positive PCR had a negative antibody test, raising the sensitivity to 87.5%; however, the numbers in this subset at BSUH were very small.

Of cases with a negative antibody test at WSHT, the RLUs were not significantly different in those with or without a positive PCR (median 0.2 in both groups, p=0.102). Similarly, at BSUH the COI was comparable (median 0.09 in both groups, p=0.162). This demonstrates that the false negatives did not have a significantly higher raw RLU or COI in either cohort than the true negatives. Of those with a positive antibody result, the RLUs were significantly higher in those with a positive PCR, compared with a negative, at WSHT (median 4.86 versus 4.05, p<0.0001). The same was not true at BSUH (41.7 for positive PCR versus 60.7 for negative PCR, p=0.213).

In the WSHT cohort, antibodies were detected up to 213 days after symptom onset with a median of 135 days (interquartile range [IQR] 126–168). By contrast, in the BSUH cohort, antibodies were detected in PCR-proven individuals up to 134 IQR test, with a median of 91 days (IQR 73–113) and in those with symptoms without a PCR test up to 195 days post-symptom onset.

Factors significantly associated with antibody positivity are described in Table 2. Male sex was associated with an increased rate of seropositivity by approximately 30%. While age was a significant finding, the overall effect size is negligible. South Asian ethnicity demonstrated in the order of three times the risk of seropositivity, with Black and East Asian ethnicity also conferring an increased risk. The ‘medical’ occupation category had the highest proportion of South Asian (8.8%), and Black staff (2.8%). Allied health professionals and nursing staff were high-risk, with medical staff being the lowest risk of front-line workers. The main hospital and local hospice had far higher risk of seropositivity than other workplace categories. Community, mental health, general practice and staff working in an elective surgical facility had a comparably low association with antibody positivity.

Discussion

This study provides real-world data on the efficacy of both Abbott and Roche SARS-CoV-2 antibody assays. Despite the higher incidence within the WSHT cohort of PCR-proven COVID-19 infections, a higher case rate in the region served by WSHT (Fig 1), and similar symptom reporting, a far lower incidence of antibody positivity is seen at WSHT (7.9% versus 13%). Notably, the majority of staff who had a prior positive PCR with Abbott actually had a negative antibody test (60.2%). This figure was slightly improved in those who also reported symptoms, with 54.7% demonstrating antibodies at WSHT and 87.5% at BSUH.

The RLUs and COI for Abbott and Roche were not significantly different for true negatives versus false negatives, so the chosen threshold was unlikely to have contributed significantly to the sensitivity of the tests. The RLUs were significantly higher for the Abbott test in those with a PCR-proven infection. However, this was not observed with the Roche assay and there are a number of confounding factors which may influence this observation so this must be interpreted with caution. There is no convincing literature to suggest that either assays are quantitative.

There is a growing collection of published healthcare worker antibody data from across the world, with a wide range of seropositivity, from <1% in Japan,^13,21 <3% in Germany²² to up to 30% in the UK,^8,10,12 Pakistan,⁹ and Sweden.¹¹ However, the large disparity between two NHS trusts under the same management and serving the same region of the UK was unexpected. The overall differences in seroprevalence between the test sites in this study may be due to differences in hospital structure, provision of PPE, caseload and availability of PCR testing for staff. However, the false-negative rate in both cohorts and the higher burden of disease in the WSHT region raises the suspicion that the overall seroprevalence may have been under-estimated, and most significantly so in the Abbott assay cohort.

Test performance of antibody assays were determined and validated using symptomatic, hospitalised patients.²³ A meta-analysis of diagnostic accuracy tests for SARS-CoV-2 serology found sensitivities ranging from 14.4–100% depending on serological test method and immunoglobulin class.²⁴ A general population study of 1,862 people in Austria found IgA antibodies in 11% whereas IgG in only 1.9%.²⁵ Discordance between assays has been reported,²⁶ and one UK study found that 58% of Abbott-negative samples demonstrated other SARS-CoV-2 antibodies.²⁷

The generalisability of antibody testing to healthcare workers with variable viral exposure (and presumably low overall seroprevalence) remains unknown, despite many thousands having been undertaken in the UK to-date. Therefore, the mass testing of healthcare workers has been controversial.^4,5 In this cohort, antibody tests were performed 61 days at BSUH and 97 days at WSHT post-symptom onset. More recent data have emerged which suggest that antibody responses in mildly symptomatic or asymptomatic individuals decline after 1 month,²⁸ and therefore within both cohorts the window for serological testing may have been missed. However, the evidence is conflicting, and studies have seen individuals with persistent antibodies beyond 6 months.²⁹ As those with a positive PCR in this study represent symptomatic individuals, even if some cases were beyond the window for when IgG would still be detectable it would be expected to be greater than 40% in any case. We saw cases with positive antibodies up to 213 days post-symptom onset. Importantly, it is clear that not all individuals will seroconvert despite confirmed SARS-CoV-2 infection, with younger age and less severe infection being associated with a lower probability of seroconversion. This represents many healthcare staff. Therefore, whether current SARS-CoV-2 antibody assays can provide reliable epidemiological data is largely undetermined.

During the pandemic, finances, resources and supply chains are precious. In the UK, mass-vaccination programmes, general population antibody tests using the lateral flow immunoassay (LFIA)³⁰ and lateral flow antigen assays for healthcare workers are underway. The SIREN study aims to serially test 100,000 healthcare workers across the UK to monitor antibodies over time. Therefore, there is unmet need for high-quality side-by-side assessment of antibody and antigen tests in the real-world environment in order to capture accurate epidemiology and vaccine response. Of note, neither the Abbott nor Roche tests will identify antibody responses to the Pfizer-BioNTech and Oxford-AstraZeneca vaccines that use the spike protein to generate immunity.

Further, in December 2020, a new and highly infectious variant of SARS-CoV-2 was identified in an individual in the UK with antibodies to SARS-CoV-2, 8 months after initial infection.³¹ A number of further mutant variants have been identified, which questions the protection from early SARS-CoV-2 antibodies. At this stage, it is not clear whether prior SARS-CoV-2 infection is protective against the new variant strains despite reassurances that the current vaccines should remain efficacious. Data on the longevity and waning of antibodies are emerging but often conflicted. With the rollout of vaccination programmes, and in the face of emerging variants, there is an urgent need to better understand the kinetics of antibody responses against SARS-CoV-2.

We also observed significant differences in workplace, occupation and ethnicity and risk of seropositivity. Given that the majority of Black and South Asian staff were in the medical profession (which was a lower risk staff category), the increased risk of antibodies is particularly striking. The reasons for this are unclear, and demand investigation, as severity of illness and risk of death is also greater in Black and ethnic minority individuals.³²

Conclusion

Serum antibody tests on healthcare workers may not accurately reflect the seroprevalence in this population and are likely to have underestimated the true incidence of SARS-CoV-2 infections. In our real-world dataset presented here, we find the two most widely used antibody tests in the UK, Roche and Abbott, have a real-world sensitivity level of 81% and 39.8% respectively. More research, based on strict criteria of appropriate timing and indication, is urgently required to establish the true validity of different SARS-CoV-2 antibody assays in real world settings.