Do different patient populations need different early warning scores? The performance of nine different early warning scores used on acutely ill patients admitted to a low-resource hospital in sub-Saharan Africa
ABSTRACT
Background Early warning scores (EWS) generated in a developed healthcare setting may not perform as well in low-resource settings in sub-Saharan Africa.
Method The performance of EWS used in developed world was compared with those generated in low-resource settings in sub-Saharan Africa.
Results When tested on 1,266 acutely ill patients consecutively admitted to a low-resource Ugandan hospital there was no statistical difference in the performance of any of the EWS tested. The performance of all the scores appeared to be improved by the addition of mobility assessment. Although statistically insignificant, the National Early Warning Score with extra points added for impaired mobility had the highest discrimination and sensitivity.
Conclusion There were only marginal and no statistical differences in the performance of EWS generated in low- and high-resource healthcare settings in a cohort of unselected acutely ill medical patients admitted to a low-resource hospital in sub-Saharan Africa.
Introduction
It has been suggested that early warning scores (EWS) generated in a developed healthcare setting may not perform as well in low-resource settings in sub-Saharan Africa.1 Since patient age, disease processes and diagnoses may be different in sub-Saharan Africa compared with more affluent patient cohorts, it cannot be assumed that the physiological response to disease will remain the same.
Two aggregate weighted EWS scores derived from sub-Saharan patients treated in low-resource settings have been reported: one of these scores includes HIV status as a predictor and the other included impaired mobility.1,2 This study of consecutively admitted acutely ill patients to a low-resource hospital in Uganda compares the performance of both these scores with the early warning score used as part of the South African triage process, as well as other scores, including the Modified Early Warning Score (MEWS), the National Early Warning Score (NEWS) and its recently introduced modification (NEWS2): all these scores are now commonly used in the Europe, North America and Australasia.3–6 It also examined two other scores: one contained only five simple parameters and the other was a binary version of NEWS.7,8 In addition, the inclusion of mobility on EWS performance was examined.
Methods
Study design
Prospective observational study carried out as part of an audit in an ongoing quality improvement project.
Setting
The setting was the 46 bed medical ward of St Joseph's Kitovu Health Care Complex, a 220 bedded healthcare facility located near Masaka, Uganda, 140 km from the capital city of Kampala. Together with the 330 bed Masaka Regional Referral Government Hospital, it serves Masaka Municipality (population of 79,200) and Masaka District with a rural population of 804,300. The hospital has no intensive care or renal dialysis unit and cannot provide mechanical ventilation, but supplemental oxygen is available from oxygen concentrators.
Participants
The participants were all consecutive acutely ill medical patients admitted during the study period and reassessed at least once after admission.
Data collection
Date were collected from 13 February 2018 to 29 April 2019, the clinical status and vital signs of every patient admitted to the hospital's medical unit were entered on admission into a clinical data management and decision support system (Rapid Electronic Assessment Data System (READS; Tapa Healthcare, Dundalk, Ireland)).
All the patients admitted were acutely ill, and no patients were excluded from the study. No patients had suffered from recent trauma. The READS bedside assessment requires that the patient's contemporaneous mental status, functional status and complaints are entered each time the vital signs are measured.9 The patient's mental status was assessed as being alert and calm, agitated and/or confused, responding to voice, responding only to pain or being unresponsive.10 Impaired mobility on presentation was defined as lack of a stable independent gait when first assessed.11 Therefore, all patients who had an unstable gait, needed help to walk, needed a wheelchair or were bedbound were deemed not to have a stable independent gait.
The final disposition of patients on hospital discharge was also recorded. All data entered into the READS system are automatically time and date stamped; it is impossible to complete a READS assessment without entering all the data required and to enter values that are outside a plausible range.
Outcomes
In-hospital death within 7 days of admission.
Data analysis
Nine aggregate weighted scoring systems were tested. Each score and the variables they require for their calculation are shown in Table 1 (a detailed description of all the scores tested are provided in supplementary material S1). The South African Triage Scale's Triage Early Warning Score (TEWS); tachypnoea, oxygen saturation, temperature, alert, loss of independence (TOTAL) score; and blood pressure, age, respiratory rate, loss of independence and peripheral oxygen saturation (PARIS) score all assessed mobility.1,3,7 For the purpose of this analysis, those patients with a stable independent gait were considered to be ‘walking’ and scored zero TEWS points, whereas those who needed help to walk or required a wheelchair scored one TEWS points, and those who were bedridden scored two TEWS points. The TOTAL score defined impaired mobility as ‘loss of independence’ (ie the inability to stand unaided), whereas the PARIS score defined it as an inability to get into bed without assistance; for the purpose of this analysis ‘loss of independence’ was considered to be the absence of an independent stable gait. In order to test if the addition of immobility improved the performance of scores that did not contain it as a predictor variable we assigned impaired mobility (ie absence of a stable independent gait) an arbitrary value of three extra points for MEWS, MEWS plus age, NEWS, NEWS2 and universal vital assessment (UVA), and one extra point for the binary NEWS.
NEWS2 is a recently introduced modification of NEWS. It provides two different scales for the assessment of oxygen saturation; the second scale is recommended for patients confirmed to have hypercapnic respiratory failure on blood gas analysis. This analysis only used the first scale (ie the one used in NEWS) as blood gas analysis was not available in Kitovu Hospital. NEWS2 also requires the recognition of new-onset confusion, disorientation and/or agitation.6 Since it was often impossible to know for certain if patients presenting to Kitovu Hospital had new-onset confusion, this analysis assigned all patients who were agitated and/or confused three NEWS2 points.
Statistical methods
All calculations were performed using Epi-Info version 6.0 (Centres for Disease Control and Prevention, Atlanta, USA). The p value for statistical significance was 0.05 and was tested using Student's t-test and χ2 analysis that applied Yates' continuity correction. Data are presented as mean (standard deviation (SD)), quartiles or number (percentage) as appropriate. The optimal cut-point for each score was that for which the sum of sensitivity and specificity was highest. Receiver operating characteristic (ROC) curves were constructed and the area under the curves compared according to the method of Hanley and McNeil.12
Ethical approval
Ethical approval of the study was obtained from the Ethics Committee Kitovu Hospital, which conformed to the principles outlined in the Declaration of Helsinki.13 Since no interventions were additional to the usual standard of care the need for written consent was waived. The study is reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology statement.14
Results
During the study period 1,266 patients were admitted to hospital for an average length of stay of 4.8 days (SD 3.8, range 0–30 days), 89 (6.7%) died while in hospital and 72 (81%) of those patients died within 7 days of admission. Patients who died within 7 days were older, had shorter lengths of hospital stay and, on admission, had higher respiratory and heart rates, lower oxygen saturations, and significantly higher EWS (Table 2). Death within 7 days of admission was associated with the use of supplemental oxygen, impaired consciousness and impaired gait, but not with gender or HIV infection (Table 3).
Although NEWS supplemented with three extra points if the patient had an impaired gait had the highest area under the ROC curve and MEWS the lowest, there was no significant difference in the performance of any of the EWS tested. However, the performance of all the scores appeared to be improved by the addition of mobility assessment, and if gait assessment was removed from TEWS its performance worsened. NEWS with three extra points for mobility that scored eight or more points had the highest sensitivity and a TOTAL score of two or more points the highest specificity (Table 4).
Discussion
Main findings
This small study found only marginal and no statistical differences in the performance of aggregate weighted early warning scores generated in low- and high-resource healthcare settings in a cohort of unselected acutely ill medical patients admitted to a low-resource hospital in sub-Saharan Africa.
Although NEWS with mobility assessment is marginally the best performing score, other scores may be more suitable in low-resource settings as they are simpler, require less equipment, are easier to remember and calculate and, by collecting additional information that is needed in the local clinical environment, may be more likely to prompt a timely and effective response.
Limitations and strengths
This study was performed in a single centre and was further limited by its relatively small size, which only examined an acutely ill cohort of consecutively admitted medical patients with a 7-day in-hospital mortality of 5.4%. We did not follow up patients after discharge, so the number of patients who may have died after discharge but within 7 days of admission is unknown. The two sub-Saharan African cohorts from which UVA and TOTAL were derived had much higher mortality rates and contained patients referred to tertiary centres.1,2 EWS may not perform the same in intensive care unit (ICU) patients as in general ward patients so it is, therefore, possible that both UVA and TOTAL would perform better, and NEWS worse, in a sicker patient population.15 However, we believe that our unselected cohort of consecutive hospital admissions is representative of patients usually encountered in most sub-Saharan African secondary care hospitals.
We arbitrarily selected mortality observed within 7 days of admission to compare the performance of different scores, even though EWS are used in clinical practice to detect imminent death within 24 hours.16 This decision was forced on us because this study only included patients who survived to admission, and did not include moribund patients who died within minutes of arrival at the hospital. As a result, a complete set of vital signs was recorded on only three patients who died within 24 hours of admission. While the estimation of mortality within 7 days has little clinical value, the purpose of this study was to compare the performance of different scores with each other, not to assess the clinical value of their predictions. Although some score components, such as blood pressure, may only change hours before death, our findings probably give a reasonable indication of what the relative performances of the scores over a shorter time period are likely to be.
Interpretation
The few studies performed in low- and middle-income countries have shown a wide variation in the performance of EWS.1,17,18 In addition to patient population, the performance of any EWS must also depend on the accuracy of the vital sign recordings used to calculate it. In many studies complete sets of vital signs were often not performed, and/or could not be measured because the equipment required was not available.17–19 In this study a considerable amount of care with vital sign measurement was taken. The READS computer system has been used at Kitovu to collect vital signs at the bedside since 2016, and for this study an electrocardiograph was used to assess heart rate, a computer application to accurately determine respiratory rate measured over 60 seconds, and a well-validated blood pressure machine to record blood pressure.20–22
Since our patients were younger than most patient populations in the developed world, it is not surprising that adding age weighting to MEWS did not improve its performance since the weightings for age would have applied to so few patients.4
The mean temperatures and systolic blood pressures of patients who lived and died were identical; changes in temperature may be intermittent and change in blood pressure may be a late sign of deterioration. Less than 10% of our patients received supplemental oxygen and more than a quarter of them died. This was because many patients in Uganda only consent to oxygen treatment when they are in extremis, as they believe the need for oxygen therapy is a harbinger of death. As a result, this widely held belief has become a self-fulfilling prophecy.
This study confirms the prognostic value of mobility assessment, as adding it to all the scores that did not already include it improved their performance.23 In the past, the standard of care was to assess the gait of acutely ill patients in order to rule out tabes dorsalis and/or Wernicke's encephalopathy. This aspect of the routine examination is now often neglected in the developed world, as many patients while being brought to hospital by ambulance are often needlessly placed on a stretcher, on which they may remain throughout their assessment. In Kitovu, only a minority of patients arrived by ambulance, so assessment of gait was unavoidable. Although various definitions of gait impairment and independence have been suggested, in practice it was easy to determine if patients needed help to mobilise or if they were immobile and required a stretcher.1,3,7
General clinical relevance
The score on which the UK NEWS is based was the best performing score of 33 other aggregate weighted ‘track and trigger’ systems tested.16,24 NEWS has also been shown to have a greater ability than these 33 scores to discriminate patients at risk of the combined outcome of cardiac arrest, unanticipated ICU admission or death within 24 hours.25 All these scores were developed in the developed world. Although it seems self-evident that risk scores should work best in the populations from which they were derived, our findings suggest that NEWS with mobility assessment was probably the best performing score, and a larger study would almost certainly confirm this statistically. This supports the concept that the human physiological response to acute illness is constant, and not greatly influenced by specific diagnoses. Although there may be modest improvements obtained by tailoring predictive scores to specific conditions, there hardly seems to be a need to develop a score for every diagnosis.
The use of a single international scoring system would simplify the training of clinical staff and might harmonise management protocols. An additional benefit could be for the benchmarking of outcomes and the quality of care. However, even though changes in vital signs, mobility and mental status may predict mortality equally well in different patient populations, this does not mean that there are not significant differences in the way different patient populations should be managed. One of the deficiencies with EWS scores is that they do not provide any insight into aetiology and appropriate treatment. In sub-Saharan Africa, for example, HIV status will greatly influence management and subsequent outcome. Therefore, it is entirely appropriate that it should form part of any triage system, even though it was not found to be a predictor of in-hospital mortality in this particular study. Similarly, the recent modifications to NEWS in the UK (ie NEWS2) have been introduced to address the needs of a small number of patients at risk of hypercapnic respiratory failure.6 In view of the local concerns about oxygen therapy this modification is not required in Uganda. Furthermore, simpler scoring systems, such as TOTAL, may require less equipment and be easier to remember and calculate and, therefore, less prone to error. Selecting which EWS to use, therefore, depends on the resources and skills available, as well as the processes of care required in different locations and patient populations, and not just on the need for accurate outcome prediction.
Conclusion
This study found only marginal and no statistical difference in the performance of EWS generated in low- and high-resource healthcare settings in a cohort of unselected acutely ill medical patients admitted to a low-resource hospital in sub-Saharan Africa. Although this small study in one hospital suggests that there may be a common human physiological response to acute illness, and that NEWS with mobility assessment is probably the best performing score, the widespread adoption of NEWS as a universal international scoring system cannot be justified. Many more confirmatory studies are required, and the choice of EWS should also depend on the processes of care required in different locations and patient populations, and not just on predictive performance.
Acknowledgements
The authors wish to acknowledge and thank Tapa Healthcare, Dundalk, Ireland, for the complimentary use of their Rapid Electronic Assessment Data System (READS). They also acknowledge and thank Alan Murray, Passage West, Ireland, for his generous donation of IT support.
Conflicts of interest
John Kellett is a major shareholder, director and chief medical officer of Tapa Healthcare, Dundalk, Ireland.
Supplementary material
Additional supplementary material may be found in the online version of this article at www.rcpjournals.org:
S1 – Detailed descriptions of the scores tested
- © Royal College of Physicians 2020. All rights reserved.
References
- ↵
- Wheeler I
- ↵
- Moore CC
- ↵
- Gottschalk SB
- ↵
- ↵
- Royal College of Physicians
- ↵
- Royal College of Physicians
- ↵
- Brabrand M
- ↵
- Jarvis S
- ↵
- ↵
- ↵
- Kellett J
- ↵
- ↵
- ↵
- ↵
- Kellett J
- ↵
- ↵
- Kruisselbrink R
- ↵
- Beane A
- ↵
- Cummings MJ
- ↵
- Opio MO
- ↵
- Rimbi M
- ↵
- Vousden N
- ↵
- Brabrand M
- ↵
- ↵
Article Tools
Citation Manager Formats
Jump to section
Related Articles
- No related articles found.