ABSTRACT
Background Identifying older people with clinical frailty, reliably and at scale, is a research priority. We measured frailty in older people using a novel methodology coding frailty syndromes on routinely collected administrative data, developed on a national English secondary care population, and explored its performance of predicting inpatient mortality and long length of stay at a single acute hospital.
Methodology We included patient spells from Secondary User Service (SUS) data for those ≥65 years with attendance to the emergency department or admission to West Middlesex University Hospital between 01 July 2016 to 01 July 2017. We created eight groups of frailty syndromes using diagnostic coding groups. We used descriptive statistics and logistic regression to explore performance of diagnostic coding groups for the above outcomes.
Results We included 17,199 patient episodes in the analysis. There was at least one frailty syndrome present in 7,004 (40.7%) patient episodes. The resultant model had moderate discrimination for inpatient mortality (area under the receiver operating characteristic curve (AUC) 0.74; 95% confidence interval (CI) 0.72–0.76) and upper quartile length of stay (AUC 0.731; 95% CI 0.722–0.741). There was good negative predictive value for inpatient mortality (98.1%).
Conclusions Coded frailty syndromes significantly predict outcomes. Model diagnostics suggest the model could be used for screening of elderly patients to optimise their care.
Introduction
There is a global acceleration in population ageing.1 For some people, this demographic shift is associated with a change in health profile, with an increased number of comorbidities, functional dependence and disability as well as costly health and social care utilisation. In the secondary care setting, older people comprise a substantial proportion (60–70%) of overall hospital inpatient bed occupancy, and have the highest readmission rates, highest rate of use of long-term care after discharge and longest lengths of stay.2,3
As registry and case-note data are not readily accessible or are unsuitable for risk stratification of older people in the secondary care setting, there is interest in utilising administrative datasets from admitted patients for this purpose.4,5 In England, large national studies explored using routinely collected operational data in the form of Hospital Episode Statistics (HES) to quantify clinical frailty, and demonstrate its association with adverse outcomes.6,7 It is yet unknown how this model could perform at a meso-level; can the model be used for an intended purpose of risk stratifying patients at the level of care provision, such that hospital managers and clinicians can plan resources to provide optimised care?
The Older people's care in acute settings: National report published by the NHS Benchmarking Network found that only 66% of trusts surveyed had a recognised frailty tool or pathway in use, 52% had a frailty unit and 40% had a dedicated geriatric team located in the emergency department, with 35% reporting no availability at weekends.8 A reliable and clinically significant measure of frailty may aid in assessing local clinical burden, guide improved access to patient services, aid in appropriate resource allocation and optimise costing accuracy. We explored the feasibility of measuring clinical frailty, in the form of frailty syndromes, using the methodology developed at national level, and explored the performance of predicting meso-level adverse outcomes.
Methods
Data source
NHS secondary care healthcare providers collect administrative and clinical information locally to support the care of the patient and delivery of NHS healthcare services. The data is then submitted to NHS Digital for processing and is returned to healthcare providers as the Secondary Uses Service (SUS) data. SUS is a rich patient-level database. It collects variables such as demographics, diagnostic codes, procedure codes and outcomes. This is predominantly an administrative dataset, primarily used for operational needs such as reimbursement and activity data, and often accessed by clinical commissioning groups. At pre-arranged dates during the year, SUS extracts data from its database to be submitted to the HES database, where the data is cleaned and validated via processes to ensure data quality, prior to storage in a data warehouse for further analysis. We utilise SUS data from West Middlesex University Hospital (WMUH) extracted for secondary analysis.
Study population
Patient records were included in the analysis if they fulfilled the criteria of age ≥65 years, attendance to the emergency department or admission to WMUH and discharged from WMUH between 01 July 2016 to 01 July 2017. Elective and non-elective admissions were included. There were no other exclusions.
Coding frailty and other variables
Each patient record corresponds to a spell covering a patient's total length of stay in hospital. In keeping with the national method, nine groups of frailty syndromes (dementia, delirium, mobility problems, falls and fractures, pressure ulcers, incontinence, functional dependence, senility, and anxiety and depression) were coded within International statistical classification of diseases and related health problems: ICD-10 diagnostic coding groups, and within all available diagnostic fields.6,7,9 Other variables included in the multivariable risk model include age, gender and number of admissions in preceding 12 months, as well as outcomes (Table 1).
Predictor and outcome variables for risk prognostic model
Descriptive statistics
Weekly trend analysis of coded frailty syndromes were plotted to explore frequency of coding. Control charts are graphical analyses that quantify expected variation in data sets and identify unusual variation through objective rule sets. They are particularly suited to time series analysis. XmR charts are a class of control chart that can be used with a wide range of data, we used a standard ruleset to identify any unusual variation.10 Trend analysis of outcomes for inhospital mortality was plotted using this methodology. Descriptive statistics exploring distribution of hospital length of stay were calculated. All analyses were conducted using MS Excel 2016.
Risk models
Missing data analysis was conducted prior to model development. Binary logistic regression using the predictor variables were run for all two outcomes separately, with bootstrapping on 1,000 resamples. Multicollinearity between predictor variables was investigated by variance inflation factor (VIF), where VIF scores of over 3 were taken to denote unacceptable collinearity.11 The Hosmer–Lemeshow statistic was calculated for each model to ascertain model performance. We plot the outcomes by risk model deciles to allow for determination of model discrimination.6 The area under the receiver operating characteristic curve (AUC) was plotted from predicted probabilities to explore predictive power.
A sensitivity analysis for age, gender and admissions in the previous 12 months was undertaken to compare the predictive power of added frailty syndromes to a baseline demographic model. Model diagnostics including negative and positive predictive value and likelihood ratios were calculated at optimal cut-point defined by Youden's index (optimal sensitivity and specificity). Statistical analysis was undertaken using the SPSS Statistics 24 package. Model diagnostics was calculated using MS Excel 2016.
Results
There were 17,199 patient spells included in the analysis: 10,804 (62.8%) were admitted under general medicine and the medical specialties and 11,565 (67.2%) were non-elective admissions. For a summary of the included patient spells by admission specialty and status, see supplementary material S1.
Coded frailty syndromes and outcomes
Overall, between 01 July 2016 to 01 July 2017, frailty syndromes were coded within SUS at WMUH with dementia at 17.7%, falls and fractures at 16.6%, mobility problems at 14.8%, anxiety and depression at 9.4%, delirium at 7.8%, pressure ulcers at 6.2%, functional dependence at 3.8%, incontinence at 2.8% and senility at 0.4%. As frequency of coding for senility was very low, it was not included in the risk models. There was at least one frailty syndrome present in 7,004 (40.7%) patient episodes.
In-hospital mortality for this population over the period was 4%. Weekly trend analysis for in-hospital mortality were conducted and displayed some seasonal peaks, but no excess variation. Mean and median length of hospital stay were 5 days and 1 day, respectively. The distribution of length of hospital stay was very positively skewed, confirmed by skewness and kurtosis statistics. As a result, upper quartile length of hospital stay was chosen as a more realistic and contextual outcome, rather than an arbitrary cut-point. See supplementary material S1 for the weekly trend analysis for frequency of coding for frailty syndromes, inpatient mortality and descriptive analysis for hospital length of stay.
Risk models
In-hospital mortality
Coded age on admission, delirium, dementia, mobility problems and pressure ulcers remain significant predictors of in-hospital mortality following multivariable regression, with coded functional dependence, falls and fractures, and admissions in last 12 months displaying a significant protective effect for in-hospital mortality. Table 2 displays the odds ratios (ORs) following multivariable adjustment for in-hospital mortality, AUC 0.738 and 95% confidence interval (CI) 0.718–0.757; see supplementary material S2 for model diagnostics in an expanded Table 2.
Odds ratio of multivariable regression model for in-hospital mortality
Upper quartile length of hospital stay
All coded frailty syndromes and age on admission remain significant predictors of upper quartile length of hospital stay following multivariable regression. Admissions in last 12 months displayed a significant protective effect following multivariable regression. Table 3 displays the ORs following multivariable adjustment for in-hospital mortality for upper quartile length of hospital stay, AUC 0.731 and 95% CI 0.722–0.741; see supplementary material S2 for model diagnostics in an expanded Table 3.
Odds ratio of multivariable regression model for upper quartile length of stay
Predictive power
Coded frailty syndromes add to predictive power for inpatient mortality and upper quartile length of hospital stay over a baseline demographic model (Table 4).
Area under the receiver operating characteristic curve of multivariable risk models by outcome
Performance metrics
All models displayed significance at p<0.05 for the Hosmer–Lemeshow statistics for goodness-of-fit tests. However, these findings have been similarly described by others who have produced models on large data set as the test is recognised to detect unimportant differences.12,13 Plotting the observed and expected outcomes by an individual model's risk deciles allows visualisation of its discrimination, and this confirms the risk models have discriminatory power. The predictor and outcome variables did not display unacceptable collinearity. All the correlations between predictors, by Spearman method, were <0.7. See supplementary material S1 for bar charts of the risk models for in-hospital mortality and upper quartile length of hospital stay ordered by model risk deciles, as well as collinearity statistics for all variables included in the risk models. Missing data analysis confirmed there was no missing data for all risk models.
Discussion
In the clinical setting of secondary care, risk stratification for older people has often used age, comorbidities, illness severity, biomarkers, functional dependence and prognostic understanding of certain diagnoses (eg advanced malignancy) to differentiate between individuals for clinical decision making. More recently, operationalised frailty scores have been used for this purpose.14,15 These scores often rely on subjective clinician judgement, parameters of physical performance, require specialised research apparatus or large amounts of data, rendering them less suitable and feasible in the secondary care setting (particularly acute care).16 Furthermore, validation for the scores has often been based on small single centre studies.17,18
Administrative datasets have characteristics that are attractive for secondary utilisation for risk adjustment. They are routinely collected with informatics standards for data cleaning and validation. In the UK, the HES is a mature database, having collected data for many years with almost complete coverage. The data is collected at patient level, allowing longitudinal linkage and analysis. However, there are recognised limitations. The data are collected at discharge and their retrospective nature limits one's ability to utilise it for contemporaneous identification of risk. Reliability of coding has been challenged, and the fields collected are limited compared to clinical datasets.19 However, a 2007 study has shown comparable performance to clinical datasets when used for risk adjustment.20
Risk adjustment models utilising secondary analysis of secondary care administrative data predominantly use demographic and diagnostic codes to differentiate risk.21–25 In the USA, there have been efforts to group frailty related diagnostic codes for risk adjustment. Examples include the Johns Hopkins ACG System (Johns Hopkins University, Baltimore, USA) frailty-defining diagnoses indicator and the High-Risk Diagnosis for the Elderly Scale.4,5 In the UK, studies have used HES as a data source, with diagnostic groups for multimorbidity and complexity, and clinical frailty being tested in the literature.6,12,26,27 In primary care, electronic medical records have been used to generate a frailty index.28
This study found that coding of frailty syndromes within meso-level administrative data is feasible. The overall prevalence of frailty syndromes coded within the study hospitals administrative data is larger when compared to the previous national level study.7 There are several possible reasons. Firstly, the study hospital had recently participated in a national quality improvement programme to improve the care of older people presenting acutely to hospital, and thus recognition and coding of frailty syndromes may have improved. Secondly, the national study showed an improved trajectory of coding for frailty syndromes annually; this later study at a local site may have benefitted from improved coding. This phenomenon may also be reflected in the model's improved predictive power compared with the national study (AUC∼0.63).6 Supplementary material S1 summarises the differences in overall prevalence of frailty syndromes coded within this study compared to the previous national level study.
This study found that coded pressure ulcers, dementia, delirium and mobility problems are significantly associated with in-hospital mortality, while falls and fractures, admissions in the last 12 months and functional dependence are protective after multivariable risk adjustment. These associations remain significant when the cohort is restricted to non-elective patients (n=11,565). The presence of pressure ulcers is strongly associated with in-hospital mortality (OR 3.248; 95% CI 2.625–4.020). The protective effects for inpatient mortality noted above were also found when this model was developed on the English national sample.6 A possible cause is selection bias, as national quality improvement programmes focused on these syndromes (National Hip Fracture Database and Falls and Fracture Audit Programme) have shown sustained and decreasing mortality rates for this cohort over several years.29 Similarly, the strongest predictor of readmission (and therefore survival) is previous emergency hospital admission.30
All coded frailty syndromes included in the risk models are significantly associated with upper quartile length of hospital stay after multivariable risk adjustment, with coded functional dependence, delirium and pressure ulcers being the strongest predictors (OR >2). In contrast, anxiety and depression, as well as incontinence was not associated with upper quartile length of hospital stay when the cohort is restricted to non-elective patients.
Model diagnostics at optimal sensitivity and specificity find that the models have good negative predictive value (98.1% and 87.1% for the outcomes of in-hospital mortality and upper quartile length of hospital stay, respectively), suggesting a possible use for screening. Likelihood ratios in diagnostics are used to determine whether a test usefully changes the probability that a condition or outcome state exists. The likelihood ratios show mild improvement over the post-test probability for both the outcomes of in-hospital mortality and upper quartile length of hospital stay, suggesting the model adds prognostic value (positive likelyhood ratio of 2.14 and 2.09, respectively).31 The sensitivity analysis suggests that coded frailty syndromes significantly improve predictive power compared to baseline demographic information. This trend is also demonstrated when the cohort is restricted to non-elective patients with the model producing AUC of 0.686 and 0.685 for inpatient mortality and upper quartile length of stay, respectively.
This reproducible methodology for measuring frailty has several potential uses. It allows secondary care trusts to understand the clinical burden of an ageing population on healthcare services, and highlights areas for improved access and resource allocation. It potentially forms a metric to track the result of quality improvement programmes for older people in secondary care and reporting for quality accounts. It could be used as a case-registry for older people at risk for adverse outcomes and focus development of care services.
This study has strengths. It was sourced from routinely collected administrative data that was representative of the whole population aged ≥65 years at the study hospital, with no missing data. It has rigorous statistical methodology, including bootstrapping, model performance metrics to explore goodness-of-fit, sensitivity analysis and model diagnostics to explore potential uses. It represents external validation of the methodology used by a previous national English study.
However, there are limitations. This study was based on a single site, and results cannot be generalised to others without further testing, though the methodology is feasibly reproducible, and is consistent with previous findings from study of the English national cohort.6 Sampling for clinical notes to compare with coded diagnosis was not feasible, thus true accuracy of frailty syndrome prevalence is not known. The population was heterogenous (eg elective versus non-elective and medical versus surgical). The low frequency of outcomes within subgroups resulted in unstable multivariable risk prediction models, thus subgroup analysis was not always possible. Exploring several years of data may mitigate this, and represents possible future work. The risk model was not adjusted for some variables known to affect outcome (eg comorbidity effect on in-hospital mortality). In addition to limitations of utilising administrative data for risk adjustment listed above, the dataset did not contain some variables known to be associated with frailty (eg socio-environmental factors and disability), and thus the model is not fully comprehensive. However, the model represents reproducible methodology to identify a group of vulnerable older people at risk of adverse events at a meso-level.
Conclusion
Frailty syndromes are coded in routinely collected secondary care administrative data and can feasibly be used to create a risk stratification model at a meso-level. There was at least one frailty syndrome present in 40.7% of patient episodes. The presence of coded frailty syndromes significantly predicts in-hospital mortality and long length of hospital stay within this dataset, and sensitivity analysis suggests the models are additive to baseline demographics, such as age. Model diagnostics suggest that the model could be used for screening of elderly patients to thereby optimise their care.
Supplementary material
Additional supplementary material may be found in the online version of this article at www.rcpjournals.org/content/clinmedicine:
S1 – Summary of the included patient spells.
S2 – Expansion of Tables 2 and 3 including the model diagnostics.
- © Royal College of Physicians 2020. All rights reserved.
References
Article Tools
