Abstract
Background Lateral flow tests (LFT) are point-of-care rapid antigen tests that allow isolation and control of disease outbreaks through convenient, practical testing. However, studies have shown significant variation in their diagnostic accuracy. We conducted a systematic review of the diagnostic accuracy of LFTs for the detection of severe acute respiratory syndrome-coronavirus 2 (SARS-CoV-2) to identify potential factors affecting their performance.
Methods A systematic search of online databases was carried out to identify studies assessing the sensitivity and specificity of LFTs compared with polymerase chain reaction (PCR) tests. Data were extracted and used to calculate pooled sensitivity and specificity. Meta-regression analysis was conducted to identify covariates influencing diagnostic accuracy.
Results In total, 76 articles with 108,820 test results were identified for analysis. Pooled sensitivity and specificity were 72% (95% confidence interval (CI): 0.68–0.76) and 100% (95% CI: 0.99–1.00), respectively. Staff operation of the LFT showed a statistically significant increase in sensitivity (p=0.04) and specificity (p=0.001) compared with self-operation by the test subjects. The use of LFTs in symptomatic patient subgroups also resulted in higher test sensitivity.
Conclusion LFTs display good sensitivity and extremely good specificity for SARS-CoV-2 antigen detection; they become more sensitive in patients with symptoms and when performed by trained professionals.
Introduction
Severe acute respiratory syndrome-coronavirus 2 (SARS-CoV-2) is the highly transmissible respiratory virus that emerged during late 2019 and resulted in a massive global outbreak of viral pneumonia (coronavirus 2019; COVID-19) in 2020 and 2021. Since its emergence, various testing methods have been developed for the efficient diagnosis of COVID-19 to limit spread through isolation and quarantine of infectious patients.1
A lateral flow test (LFT), or lateral flow immunoassay, is a device that is intended to detect the presence of a target substance, such as a biological antigen, as a point-of-care test. More recently, they have emerged as one of the most important tests for the diagnosis of SARS-CoV-2 infection. The test works via binding of conjugated antibodies to a specific target antigen in a sample and this antigen–antibody complex (positive test) is evident on the test strip through a coloured line.2
Although the gold standard for diagnosis of COVID-19 remains the reverse transcriptase polymerase chain reaction (RT-PCR), it is more expensive, requires trained professionals for processing and has a longer time for results to show.3 By contrast, LFTs have a quick turnaround time, are easily available, have relatively lower costs and require limited training, allowing their use by the test subject.4 It is because of these favourable characteristics that LFTs have become popular worldwide and are considered a powerful tool to tackle the COVID-19 pandemic.5
Consequently, mass testing for COVID-19 with LFT kits was rolled out in the UK and many other countries in 2021. In the case of a positive test result, the affected individual was required to self-isolate for a period of time. This rule was meant to reduce transmission and positive caseloads. However, the actual outcome of such rules is highly dependent on the test performance and disease prevalence in the region. In fact, if the tests perform poorly, these rules could have a negative impact on various services, including healthcare and the overall economy.6
Research has shown significant variation in the reported sensitivity of different lateral flow assays.4 Our main objective here was to conduct a systematic review and meta-analysis to identify the sensitivity and specificity of these tests based on published data, and also to identify various factors that could affect the performance characteristics of the lateral flow devices with regards to the detection of COVID-19.
Methodology
Study design
This was a systematic review of clinical studies published in peer-reviewed journals. This followed the guidance laid out in the Cochrane handbook for systematic reviews of diagnostic test accuracy.7
Search strategy
Literature search strategies were developed using medical subject headings (MeSH) and text words related to the title. The search was performed in the Medline, EMBASE, Scopus and Cochrane databases using various combinations of keywords and subject headings: ‘Covid 19’, ‘SARS-CoV-2’, ‘Coronavirus’, ‘antigen test’, ‘antigen detection’ and ‘lateral flow devices’.
To ensure maximum capture, we scanned the reference lists of included studies or of the relevant reviews that had been identified through the search for potential studies. We included studies that were published until the 31 January 2022. Four reviewers reviewed each paper generated from the search and excluded articles first based on the abstract and then on reviewing the full text according to the inclusion and exclusion criteria and relevance to the topic (Fig 1).
PRISMA diagram demonstrating the study selection process.
Study selection
The inclusion criteria were as follows:
prospective and retrospective comparative cohort studies, case-controlled studies, cross-sectional studies, and randomised controlled trials;
studies with both adult and paediatric populations;
published in English, peer reviewed and available as full text in the medical database;
COVID-19 confirmed through PCR testing as the reference standard;
have sufficient data to calculate true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN);
used commercially available antigen detection kits.
The exclusion criteria were as follows:
studies did not have a control group;
studies were themselves published as systematic reviews/meta-analyses.
Quality assessment and data selection
The QUADAS-2 tool was used to assess the quality of the included studies. This was undertaken by one reviewer and double-checked by another. The studies were assessed against four domains: patient selection, index test, reference standard and patient flow and timing; the risk of bias was graded as low, unclear or high. Any disagreements were resolved by discussion. Publication bias was assessed through Deeks' regression test and funnel plot.8
Data synthesis and analysis
The following data were extracted from the papers: country of study and geographic region; period of study and season in that country; antigen test kit used; specimen collected and who collected by; antigen test results (TP, TN, FP or FN); result sensitivity; result specificity; total population and those confirmed to have had COVID-19; and population characteristics with regard to symptoms. R software was used for all statistical analyses.9
Where studies had reported only sensitivity and specificity, the TP, TN, FP and FN were calculated from precision estimates and their respective confidence intervals (CIs), to produce a 232 contingency tables. Descriptive forest plots were produced to illustrate the sensitivity, specificity and their corresponding CIs using the package ‘meta’.10 Bivariate analyses were performed using the package ‘mada’.11
Summary receiver operator curves (SROCs) with confidence region for sensitivity and false positive rate (1–specificity) were produced. The area under the curve was calculated to assess the overall accuracy of lateral flow devices. The heterogeneity was assessed through a visual distinction of forest plots and I2 test. Meta-regression analyses were performed to identify the potential covariates that contributed to the heterogeneity.
Results
Overall, the literature search identified 2,708 articles. These were checked against the inclusion and exclusion criteria, which resulted in 76 articles. A few of the studies included evaluated more than one study cohort; therefore, the total number of study cohorts included in our meta-analysis was 106, comprising 108,820 patients for analysis.
The basic characteristics of each study, comprising geographical regions, seasons, whether the patient was symptomatic, and whether LFT was self-administered, are summarised in supplementary material S1. There were 30 studies that included symptomatic patients only and seven studies included only patients who were asymptomatic. In terms of the swabbing methods, 48 reports (45%) did not mention how the tests were performed. The population size of these studies varied from 56 to 15,402, with 86 cohorts (81%) including adult patients only.
Of the study cohorts, 95 (90%) used nasopharyngeal swabs as a specimen for their antigen tests; the remaining 10% of studies had different sampling methods, including saliva sampling, bronchoalveolar lavage fluids and oropharyngeal swabs. The lateral flow devices, produced by 28 different manufacturers, varied across the 106 cohorts. The geographical distribution of the study populations, presence or absence of symptoms and the lateral flow devices used are illustrated in Fig 2.
Geographical plots demonstrating the distribution of (a) study populations, (b) the lateral flow devices used and (c) the presence or absence of coronavirus 2019 symptoms.
The quality assessment of studies as per QUADAS-2 is detailed in supplementary material S2. There was a relatively low risk of detection bias among the studies as per the second domain in the ‘Index Test’, meaning that most studies carried out interpretation of the LFTs independently from having knowledge of the PCR status of the test subject. Similarly, Domain 3, the ‘Reference Standard’, also had a generally low level of observer bias: the reference test or PCR was carried out independently without knowledge of the lateral flow result and it is well known that PCR has the greatest sensitivity and specificity of any test for COVID-19. There was a slightly higher risk of selection bias noted among studies, as seen in the first domain of supplementary material S2. Some studies did not clearly describe the characteristics of participants, and this might have an impact on diagnostic accuracy when it comes to test subjects swabbing themselves. Finally, there was also a potential for bias in terms of unclear patient flow in the last domain: ideally, studies should have carried out lateral flow testing and PCR simultaneously; however, if this was not described, it could lead to discordant results through disease state alteration with time.12 Fig 3 represents Deeks' regression test for funnel plot asymmetry and shows an absence of publication bias (p>0.05) among the studies included in the meta-analysis.
Deeks' regression test and funnel plot for the studies included in the meta-analysis. p=0.07, suggesting an absence of publication bias. The vertical dashed line corresponds to the no intervention effect and the diagonal dashed line is the regression line. The numbered black dots indicate individual studies included in the meta-analysis. LFD = lateral flow device.
Diagnostic accuracy of lateral flow devices for the detection of SARS-Cov-2 antigen
The pooled sensitivity and specificity of an antigen test for the detection of SARS-CoV-2 were 0.72 (95% CI, 0.68–0.76) and 1.00 (95% CI, 0.99–1.00), respectively. Forest plots for sensitivity and specificity are given in supplementary material S4 and S5. The AUC of bivariate SROC was 0.96 (Fig 4). There was significant heterogeneity among the studies included in the meta-analysis. This was further explored through meta-regression analysis to identify the covariates contributing to the heterogeneity. The covariates included in the bivariate meta-regression analysis were: presence of symptoms, manufacturer of the antigen test used, specimen type, operator of test and season in the country when the test was performed.
Summary receiver operator curve characteristics for lateral flow devices for the detection of coronavirus 2019 (COVID-19). Green circle indicates the 95% confidence interval. Triangles indicate individual studies.
The z values for the regression coefficients for LFTs administered by staff rather than by test subjects were significantly higher for sensitivity and lower for the false positive rate (p=0.04 and 0.001, respectively) (supplementary material S3). Similarly, the sensitivity of the LFDs was higher if the patients had symptoms. However, presence of symptoms did not have any effect on the test specificity. Therefore, our analysis implied a higher sensitivity in patients with symptoms, as reported previously.2 However, our analysis also implies a potentially higher false positive rate in test subjects performing the test themselves, which was not found in previous studies, such as that by Mistry et al.4
Studies that reported the date and place of the study conditions allowed us to extract the average temperature for the location at that time and then to compare this with the reported sensitivity to assess whether this had an effect on diagnostic accuracy. However, there was no significant correlation between temperature and sensitivity detected (Fig 5).
Scatter plot of average temperature at the time of the lateral flow test versus sensitivity of that test.
With regard to specimen type or manufacturers of the antigen test used, statistical significance was not reached in any of these domains. Hence, no evidence of any contribution from these covariates to the heterogeneity was demonstrated.
Discussion
We evaluated the diagnostic accuracy of lateral flow devices used in the detection of COVID-19 as per current published literature. In total, 76 studies and 106 cohorts were included, with a total of 108,820 patients and minimal publication bias. The findings showed that detection of COVID-19 using lateral flow devices had a pooled sensitivity and specificity of 0.72 (95% CI, 0.68–0.76) and 1.00 (95% CI, 0.99–1.00), respectively.
Our study included more test subjects and spanned more continents compared with other recently published systematic reviews.2,4 Furthermore, our data present a significant update when considering the increased burden of use of LFTs compared with earlier in the pandemic, when PCR was the main test used for patients.
Most of the studies were conducted in Europe, with fewer studies reported in the literature from countries in Africa or Asia, which could reflect ongoing global health inequalities in the COVID-19 pandemic (Fig 2).13 This is a positive of our study in that it also demonstrates the academic inequalities exposed by the global pandemic as well as the inclusion of papers among five different continents, making our review a truly global study.
There are some limitations of our study, such as the variable thresholds used to define a positive result depending on the device manufacturer and the lack of reporting of exact environmental temperatures at which the antigen test was performed. Such limitations affected our ability to quantify seasonal variation on test accuracy.
Another limitation is the heterogeneity of studies included. Nevertheless, our meta regression analysis led to interesting findings, such as the increased sensitivity and reduced false positive rate found when trained staff conducted the LFTs. Both were statistically significant results, taking into account the relatively low number of studies that reported these data. Furthermore, our study reaffirmed the increased sensitivity of LFTs in symptomatic patients; which is in keeping with other research.14
Insufficient data were available to investigate effect time after symptom onset; therefore, it is unclear when sensitivity reduces after onset of symptoms and we can only imply that the tests are more likely to be negative once symptoms abate.
Finally, based on these data, we could not demonstrate a significant variation in the sensitivity of these tests with temperature; therefore, we were unable to conclude that there was any significant seasonal variation or variation with temperature in terms of test accuracy.
Within our study, the assays shown to meet appropriate criteria, such as the World Health Organization's priority target product profiles for COVID-19 diagnostics (‘acceptable’ sensitivity ≥80% and specificity ≥97%), can be considered as replacements for laboratory-based RT-PCR when immediate decisions about patient care must be made, or where RT-PCR cannot be delivered in a timely manner.15
Conclusion
This systematic review reaffirmed that rapid antigen tests have a reasonable sensitivity (0.72) and a high specificity (1.00) that meet the appropriate criteria for COVID-19 diagnostics. Our data suggest that the test performance is better when administered by trained staff compared with those done by patients themselves. In addition, the sensitivity of LFTs is also higher in patients who are symptomatic.
Future research
It would be beneficial to consider further studies investigating the effect of symptom status or timing after onset of symptoms on the diagnostic yield of LFTs. Further clinical trials, comparing self-swabbing and staff swabbing, as well as monitoring the temperature at the time of the LFT, are also required. Addressing these factors could provide methods to further optimise the precision of LFTs.
Supplementary material
Additional supplementary material may be found in the online version of this article at www.rcpjournals.org/content/clinmedicine:
S1: basic characteristics of the included studies.
S2: quality assessment of studies.
S3: regression coefficient results.
S4: forest plot for test sensitivity.
S5: forest plot for test specificity.
- © Royal College of Physicians 2023. All rights reserved.
Jump to section
Related Articles
Cited By...
- No citing articles found.