ABSTRACT
Hospital-acquired acute kidney injury (H-AKI) is a common cause of avoidable morbidity and mortality. Therefore, in the current study, we investigated whether vital signs data from patients, as defined by a National Early Warning Score (NEWS), can predict H-AKI following emergency admission to hospital. We analysed all emergency admissions (n=33,608) to York Hospital with NEWS data over a 24-month period. Here, we report the area under the curve (AUC) for logistic regression models that used the index NEWS (model A0), plus age and sex (A1), plus subcomponents of NEWS (A2) and two-way interactions (A3), and similarly for maximum NEWS (models B0,B1,B2,B3). Of the total emergency admissions, 4.05% (1,361/33,608) had H-AKI. Models using the index NEWS had lower AUCs (0.59–0.68) than models using the maximum NEWS AUCs (0.75–0.77). The maximum NEWS model (B3) was more sensitive than the index NEWS model (A0) (67.60% vs 19.84%) but identified twice as many cases as being at risk of H-AKI (9581 vs 4099) at a NEWS of 5. Based on these results, we suggest that the index NEWS is a poor predictor of H-AKI. The maximum NEWS is a better predictor but appears to be unfeasible because it is only knowable in retrospect and is associated with a substantial increase in workload, albeit with improved sensitivity.
Introduction
Many patients admitted to hospital are at risk of acute kidney injury (AKI)1 and AKI is particularly common in patients admitted to hospital as an emergency. Acute kidney injury occurs when there is a rapid reduction in kidney function, as measured by serum creatinine or oliguria (<400 mL of urine per 24 h). Acute kidney injury can be a ‘silent’ condition with no external signs or symptoms. The National Confidential Enquiry into Patient Outcome and Death highlighted systematic failings of identification and management of AKI in hospitals and recommended that all emergency admissions should be screened for early detection of AKI.1,2 Therefore, an AKI risk assessment tool could be helpful and has been developed for use with intensive care,3–5 adult cardiac surgery,6,7 paediatric cardiac surgery8 and general surgery9,10 patients; however, no such tool has been developed for those patients admitted as an emergency who mostly stay on general wards. Although changes in serum creatinine level and urine output are indicators of risk, it is important that these biomarkers are monitored alongside a ‘track and trigger’ system1 to enable early recognition and effective treatment.
In the NHS UK, the patient’s vital signs are monitored and summarised into a National Early Warning Score (NEWS) as an integral part of the process of care and recorded on a clinical observation chart. The NEWS is mandated by the Royal College of Physicians (RCP, London) and is derived from seven physiological variables or vital signs: respiration rate, oxygen saturations, any supplemental oxygen, temperature, systolic blood pressure, heart rate and level of consciousness (alert, voice, pain, unresponsive). These data items are routinely collected by nursing staff, usually for all patients, and then repeated thereafter at a frequency dependent on local hospital protocols. National Early Warning Score points range from 0 (lowest severity of sickness) to 20 (highest severity of sickness). Studies have shown that electronically collected NEWS11 is highly reliable and accurate compared with paper-based methods.12–14
National Early Warning Score is a clinically valid measure of severity of illness and, therefore, there are grounds to hypothesise that, as a marker of the physiological response to severe illness, NEWS might predict at least a proportion of patients who subsequently develop an AKI (either as a result of the underlying physiological derangement or secondary to iatrogenic causes applied as part of their investigation or treatment). The NEWS is the first component part of a ‘track and trigger’ system primarily designed to help clinicians recognise patients who are acutely ill and to mitigate the risk of acute deterioration. The NEWS was first introduced as a predictor of risk for escalation to higher levels of care and as a predictor of mortality. Nonetheless, given the multifactorial aetiology of AKI15 and guidance that highlights deterioration as a risk factor for AKI (see chapter recommendations in NICE guidelines1), we sought to determine the extent to which NEWS, and its physiological subcomponents, could predict hospital-acquired AKI (H-AKI) following emergency medical admission. We consider two clinically relevant time points for NEWS: the first or index NEWS (obtained within ±24 h of the admission) and the maximum NEWS before the onset of AKI.
Methods
Setting and data
Our cohort of emergency medical admissions was from York Hospital, UK (managed by the York Teaching Hospitals NHS Foundation Trust), which has approximately 700 beds. It has been exclusively using electronic NEWS scoring since at least 2013 as part of its in-house electronic patient record systems. We considered all adult (age ≥16 years) emergency medical admissions discharged during a 24-month period (1 January 2014 to 31 December 2015). For each admission, we obtained the patient’s age, sex, admission and discharge dates and time stamp, AKI score and electronic NEWS (including its subcomponents). We excluded patients who were on dialysis (n=16). We excluded records where NEWS was missing or recorded outside ±24 h of the admission time because such scores are less likely to reflect the presenting sickness profile of patients who are acutely ill. The index value was defined as the first score recorded within ±24 h of the admission time, and the maximum value was defined as the highest recorded value before the onset of an AKI compared with the maximum NEWS of a patient without AKI during their stay in hospital. National Early Warning Score is also recorded in the emergency department and, therefore, can pre-date the admission date and/or time.
Our primary outcome was H-AKI. There are several definitions of AKI 16 and we adopted the national AKI algorithm used by NHS UK,17 which is based on serum creatinine changes over time and produces a four-level AKI score. An AKI score of 0 indicates no AKI, and scores of 1, 2 and 3 indicate AKI with increasing severity. Emergency admissions with AKI score 0 were classified as having no AKI and those with AKI scores of 1, 2 or 3 were classified as having an AKI. Acute kidney injury can be detected in different healthcare settings, such as community-acquired AKI (AKI is present and detected on admission to hospital or in primary care), and H-AKI (AKI occurs during a hospital stay). We defined H-AKI as that which developed >48 h after admission during that episode of hospital stay. We considered AKI developed ≤48 h before admission as community-acquired AKI.
Statistical analyses and modelling
We summarised continuous variables as means and standard deviations and used box plots to visually compare (without outliers) the distribution of continuous variables with AKI status (yes/no). However, we summarised the length of stay as the median and interquartile range (IQR) because of its skewed distribution. We considered NEWS at two clinically relevant time points: the first or index NEWS within ±24 h of admission and the maximum NEWS before the onset of AKI (compared with the maximum NEWS for patients discharged without AKI). We developed logistic regression models that used the index NEWS alone (model A0), plus age and sex (model A1), plus subcomponents of NEWS (model A2), plus statistically significant (p<0.01) two-way interactions (model A3), and similarly for maximum NEWS (models B0, B1, B2 and B3). Although the NEWS algorithm only uses the systolic blood pressure, we also used the diastolic blood pressure in our statistical models (A3 and B3).
The internal validity of these models was assessed using bootstrapping18 methods, which involved taking 100 samples (with replacement) from the data set.19 Each sample can be considered as repeating the data collection with the same number of patients and under identical circumstances as the original. In each of the 100 bootstrap samples, a regression model was estimated and evaluated on the original sample to estimate statistical ‘optimism’.19–21 This validation approach leads to better predictions of outcome for patients similar to the development population. The performance of the models was assessed in terms of discrimination, which can be quantified by the area under the receiver-operating characteristic (ROC) curve (AUC).20 The ROC curve is a plot of the sensitivity (true positive rate), versus 1-specificity (false positive rate) for consecutive predicted risks. The AUC is interpreted as the probability that a randomly selected patient with H-AKI has a higher risk score than a randomly chosen patient without H-AKI. An AUC of 0.5 is no better than tossing a coin, whereas a perfect model has an AUC of 1. The higher the AUC, the better the model. In general, values <0.7 are considered to show poor discrimination, values of 0.7–0.8 can be described as reasonable, and values >0.8 suggest good discrimination.22 The 95% confidence interval for the AUC was derived using DeLong’s method, as implemented in the pROC library.23 We assessed the calibration of each model using calibration plots.19
We determined the sensitivity, specificity and positive predictive values for the index-NEWS-only model (A0) and compared this with those models that had the highest AUC for the index NEWS (model A3) and the maximum NEWS (model B3) so that some assessment could be made of the likely impact of using these models to identify patients predicted to be at risk of AKI (yes/no) using probability thresholds from the NEWS-only model (A0). The number of admissions that would be identified as being at risk of AKI, for each probability threshold, represents the impact on workload. All analyses were undertaken in STATA24 and R25 using rms18 and pROC23 packages.
As a secondary analysis, we used survival analysis to explore how the risk of AKI (in terms of the hazard function [h(t)] varied with length of hospital stay.
Ethical approval
Ethical approval for this study was granted by the NHS Research Ethics Committee (Ref 16/HRA/2598).
Results
Cohort description
There were 36,776 emergency admissions for York hospital during the 24-month study period (1 January 2014 to 31 December 2015), of which 6.1% (2,255/36,776) were excluded because they had community-acquired AKI and 2.5% (913/36,776) were excluded because of missing NEWS, leaving 91.4% (33,608/36,776) in our analyses. The proportion of admissions with H-AKI was 4.0% (1,361/33,608), of these 1,093 were AKI stage 1, 180 were AKI stage 2, and 88 were AKI stage 3. For those patients who acquired an AKI, the median time to AKI diagnosis was 7 days. The hazard peaked at 3 days (h(t) = 0.016) and remained above 0.010 up to 7 days, thereafter steadily falling to a stable value of approximately 0.005.
Tables 1 and 2 show the profile of our cohort of emergency admissions according to H-AKI status (non-AKI/AKI). Admissions with AKI were approximately 10 years older than non-AKI admissions (AKI 77.7 years vs non-AKI 67.1 years), had longer hospital stays (AKI 25.5 days vs non-AKI 6.2 days) and higher in-hospital mortality (AKI 29.2% vs non-AKI 3.8%). Admissions with AKI also had higher index NEWS (AKI 3.3 vs non-AKI 2.4) and maximum NEWS values (AKI 6.8 vs non-AKI 4.2) compared with non-AKI. Fewer differences were seen in the subcomponents of NEWS (Table 2), with the exception of oxygen supplementation, which was higher in AKI admissions (index values: AKI 18.7% vs non-AKI 10.3%; maximum values: AKI 45.8% vs non-AKI 19.1%).
Profile of emergency admissions with respect to their acute kidney injury (AKI) status
Profile of the index and maximum National Early Warning Score (NEWS) and their subcomponents for emergency admissions with respect to their acute kidney injury (AKI) status
Fig 1 shows boxplots of NEWS and its subcomponents with patient AKI status (No/Yes) for index NEWS and maximum NEWS. For index NEWS (Fig 1A) emergency admissions who developed AKI had a higher NEWS, lower diastolic pressure, lower oxygen saturation and more-variable respiratory rates. For maximum NEWS (Fig 1B), emergency admissions who developed AKI had higher NEWS, higher respiratory rates, lower systolic blood pressure, higher pulse rates and lower oxygen saturation.
Boxplot without outliers for continuous covariates based on (a) index National Early Warning Score (NEWS) and (b) maximum NEWS.
Statistical modelling of AKI
We developed logistic regression models based on the index NEWS (models A0, A1, A2 and A3) and the maximum NEWS (models B0, B1, B2 and B3), as shown in Table 3. The lowest AUC was for the index NEWS (model A0, AUC 0.5936) and the highest was for the maximum NEWS (model B3, AUC 0.7731). Fig 2 shows the ROC plots for models based on the index NEWS (models A0, A1, A2 and A3) and the maximum NEWS (models B0, B1, B2 and B3). Fig 3 shows the calibration plots for each model. Models with interactions (models A3 and B3) were better calibrated than models without interaction effects.
Area under the curve (AUC) of all models with 95% confidence intervals
The receiver-operating characteristic (ROC) plots for estimating the risk of hospital-acquired acute kidney injury (AKI) using the National Early Warning Score (NEWS). (a) Models based on index NEWS. (b) Models based on maximum NEWS.
Internal validity of all models for predicting the risk of acute kidney injury (AKI) in hospital using the index National Early Warning Score (NEWS) (top row: models A0–A3) and the maximum NEWS (bottom row: models B0–B3) as predictors.
Sensitivity, specificity and workload analysis
Table 4 compares the sensitivity, specificity and positive preditive value (PPV) of the index NEWS-only model (A0) versus the index NEWS model (A3) with the highest AUC and the maximum NEWS model (B3) with the highest AUC for selected probablity thresholds. For instance, for a predicted probability cut-off of 0.0506, corresponding to a NEWS score of 5, the performance of the index-NEWS-only model (A0) was: sensitivity = 19.84%, specificity = 88.13%, PPV = 6.59%, and would identfy 4,099 (12.20%) patients as being at risk of AKI. At the same cut-off, the index NEWS model (A3) has sensitivity = 55.33%, specificity = 68.13% and PPV = 6.83%, and would identify 11,029 (32.81%) patients as being at risk of AKI. Likewise, at the same cut-off, the maximum NEWS model (B3) has sensitivity = 67.60%, specificity = 73.14% and PPV = 9.60%, and would identify 9,581 (28.51%) patients as being at risk of AKI. Therefore, the workload increases from 4,099 to 9,581 for an increase in sensitivity from 19.84% to 67.60%.
Comparison of workload (ie number of cases predicted to be at risk of acute kidney injury; AKI) and sensitivity analysis of index and maximum National Early Warning Score (NEWS) models at different predicted probability thresholds for NEWS 0–6a
Discussion
In this study, we found that the index NEWS and its subcomponents recorded within ±24 h of emergency admission were relatively poor predictors of H-AKI (with AUC <0.7). Our findings are in line with those of Potter et al,26 who found that potassium and bicarbonate biomakers were more responsive markers in AKI than NEWS and suggested that NEWS has little role in the escalation of patients with AKI. We did find that models based on the maximum NEWS had better performance, but their use in practice is undermined by the fact that the maximum NEWS can only be determined in retrospect and the substantial increase in the number of cases identifed at risk of H-AKI, albeit with increased sensitivity.
In 2015, the Think Kidney Risk Workstream conducted a systematic review that focused on risk scores for predicting AKI.27 They identified 12 risk tools, which included age, chronic kidney disease (CKD), cardiac and liver disease, nephrotic drugs, sepsis and abnormal vital signs.27 These risk tools used admission characteristics either at the point of hospitalisation or during hospitalisation, wherein the latter showed moderate predictive ability. However, these risk tools have yet to be externally validated.
Given the difficulties in accurately identifying patients who will develop AKI and the large numbers of patients who appear to be at risk (Table 4), an alternative approach is to consider applying blanket measures to treat AKI in all patients who are acutely ill, patients undergoing surgery, and those planned for contrast-imaging studies. Stopping nephrotoxic medications, such as angiotensin-converting enzyme inhibitors (ACEi) and angiotensin receptor blockers (ARBs), or other antihypertensive agents (where it is safe to do so suddenly, excluding beta-blockers) might reduce the overall risk of AKI, particularly in patients already at risk of hypotension. The risk:benefit ratio of stopping these medications temporarily is unlikely to be adverse in the short term unless they are part of the treatment for the primary diagnosis causing admission to hospital. Maintaining good hydration in all patients and giving more attention to older and frail patients who are less likely to be able to maintain adequate fluid intake could also reduce the risk of that cohort of patients as a whole developing AKI.
There are two key limitations to this study: (i) although NEWS is widely used in the NHS, the extent to which the results from this single-centre study can be generalised remains unclear; and (ii) there is a possibility that some patients discharged within 48 h acquired an AKI in the immediate postdischarge period. Had they remained in the hospital, they would then have been diagnosed with a H-AKI.
In summary, the index NEWS is a poor predictor of H-AKI. The maximum NEWS is a better predictor but appears unfeasible because it is only knowable in retrospect and is associated with a substantial increase in workload, albeit with increased sensitivity.
Author contributions
DR had the original idea for this work. MF undertook the statistical analyses. RH and KB extracted the necessary data frames. DR and MAE provided a clinical perspective. MAM and AS contributed to the study design and interpretation of results. MF and MAM wrote the first draft of this paper and all authors subsequently assisted in redrafting and have approved the final version.
Competing interests
The authors declare no conflicts of interest.
Funding
This study was funded by the Health Foundation. The Health Foundation is an independent charity working to improve the quality of healthcare in the UK.
Acknowledgements
The authors would like to express their gratitude to Sue Rushbrook and Gary Hardcastle at York Hospital for their support in obtaining the data frame. This study is part of the Health Foundation’s Insight 2014: Informatics in healthcare improvement award. The Health Foundation is an independent charity working to improve the quality of health care in the UK.
- © Royal College of Physicians 2018. All rights reserved.
