Collaborative critical care prediction and resource planning during the COVID-19 pandemic using computer simulation modelling: future urgent planning lessons
ABSTRACT
In March 2020, epidemiological modelling of COVID-19 predicted overwhelming demand on healthcare resources, yet data that emerged painted a different picture. Our management science health systems team at the University of Strathclyde collaborated with one NHS organisation to contextualise national policy and predict local resource needs before the pandemic took hold. Using action research, we combined organisational expertise, local and international data, and healthcare systems expertise to create a discrete event simulation model that predicted concurrent resource use over the first 10 weeks of the pandemic with realistic estimates of uncertainty. This allowed the organisation to create an effective strategy for resource planning. Had they followed national guidance, the costs would have been unwieldy and futile. Our decentralised approach delivered valuable information in a timely manner. This case study is unique in healthcare literature and serves as an example of successful methodology for similar crises.
Introduction
The coronavirus pandemic originated in China in late 2019 then spread to Europe in early 2020. The UK government adopted a policy of voluntary isolation and herd immunity until epidemiological modelling from Imperial College London (ICL) predicted that unsustainable demand on healthcare facilities would overwhelm the NHS.1 The move to a ‘lock-down’ strategy is likely to have mitigated a significant proportion of this anticipated demand. The Scottish Government initially modelled activity using the ICL model and shared the results with local organisations. They presented alarming predictions of whole population impact over a 24-week period with peak activity anticipated in mid-May. These were accompanied by wide confidence intervals and the possibility of four to five times usual critical care activity at peak, and organisations were advised to create the resource required for this ‘reasonable worst case scenario’. These predictions continued to inform the national planning message until late April despite the emerging picture of lower than predicted levels of activity from late March onwards and international patterns of activity.
Our health systems team in the University of Strathclyde, Glasgow, collaborated with a local NHS organisation in early March to make sense of national guidance and inform development of a strategy reflective of local health resource use and emerging disease activity. The organisation ran three, mid-sized, acute hospitals serving over 650,000 people (approximately 12% of the Scottish population) with a significant degree of chronic ill health compared with other regions.2
We explain why our decentralised approach using discrete event simulation (DES) modelling delivered superior guidance for local strategy and the study provides a methodological example of health system planning in emergent phenomena. We hope it will encourage other healthcare teams to collaborate with management scientists and operational researchers and realise the benefits of collaboration with health systems academics for future healthcare delivery in all settings.
Methodology
The aim of our model was to predict critical care resource requirements over the initial 10-week period of infection. The uncertainty inherent in predictive modelling and novel infections, plus international trends suggested usefulness and accuracy would diminish if we modelled over a longer time horizon. Restrictive time scales and the threat to health on a massive scale necessitated a pragmatic approach. Crucial to this work was the establishment of a team with clinical, local systems and systems-flow expertise. We felt that action research using a plan, do, study, act (PDSA) framework was the most suitable approach.3
Subject matter expertise, retrospective local data and emerging international data were used to create a mathematical model predictive of new COVID-19 cases in the region per week. These results were used to populate a DES model replicating local hospital activity through a combination of deterministic (fixed) and stochastic (random) processes. Verification and validation with emerging data was performed throughout via repeated PDSA cycles (Fig 1).
Informing the spreadsheet model
By mid-March, a large volume of information on COVID-19 dynamics was available.4,5 Data on trends and mitigation policies from the pacific region provided us with several hypothetical scenarios to consider. We assumed that lockdown measures would reduce infection spread based on the data from countries previously affected. Early pandemic research made little reference to nosocomial transmission prompting us to model new infection in the community only.
Advice from the UK and Scottish governments early in the pandemic recommended local healthcare organisations plan for 80% of their population contracting COVID-19 with 12% requiring hospitalisation (Scottish Government correspondence). At the same time, daily data published by the Italian ministry of health showed a rapid rise of new COVID-19 cases, most notably in the Lombardy region.6 Adopting a similar strict lockdown strategy to the UK, the wave of new cases in Italy peaked 2.5 weeks later, after which a decline in the rate of new cases set in.6 Analysis of the Italian ministry of health data produced a predicted maximum infection rate of 5% in the first 10–12 weeks. We applied this to UK data, adopting a 5% maximum infection rate as a plausible assumption for the Scottish population. New hospital cases per week were calculated using: total population presenting over 10 weeks = population infected x percentage at high risk x percentage of symptomatic seeking care.
We assumed upper (30%) and lower (20%) bounds of the infected population seeking healthcare for symptoms.7,8 One oversight of the national model was the assumption that hospitalisation would only occur only in patients with severe disease. No available studies described hospitalisation rates for patients with non-severe COVID-19; however, conversations with frontline clinicians revealed this to be a significant phenomenon in the UK in March and April. We included this in our model assuming that 5% of all symptomatic patients would seek emergency help via community hubs and further assuming that 20% of non-severe cases would be admitted to an acute medical bed in the first instance. Another important aspect not raised by national models was suitability for critical care in moribund patients. Policy makers in the UK made specific recommendations around hospitalisation of end-of-life patients in long-term care facilities.9 We were able to identify and remove this group from our model using census data.10 Dialogue with clinicians also determined that some hospitalised patients would be unsuitable for escalation to critical care due to severe comorbidities. We identified patients aged over 75 years as a proxy cohort to represent this group, allowing us to model outcomes specific to the local population, realistic of healthcare practice.
Finally, early case reports identified age and hypertension as dominant comorbidities in critical care patients.11–15 Local data on the age-specific prevalence of hypertension in our population identified a greater cohort of persons at risk of severe infection than age alone would suggest. We combined risk according to age with local prevalence of hypertension to create our high-risk group.1
DES model
Dynamic modelling approaches such as DES are highly advantageous in predicting bed activity. DES allows you to replicate interconnected activity over any time period and to analyse flow, bottlenecks and alternative operational scenarios. Our model of inpatient activity was developed in Simul8 software. As our organisation's initial concern was concurrent critical care activity, we largely focused on inpatient and critical care beds (Fig 2).
We created three categories of COVID-19 infection: critically severe, severe (non-critical care hospitalisation) and non-severe (hospitalisation for other concern). Patients entered the model according to a Poisson distribution reflective of emergency attendances. A separate stream of non-COVID activity was created to model patients in competition for resources. Based on conversation with clinicians, we modelled a small proportion of these patients to be incidentally COVID positive. Model entities, assumptions and distributions are described in supplementary material S1. We assumed unlimited resource to see new patients, consistency in referral decisions across all sites, and no intra-hospital transmission.
Accurate and realistic length of stay (LoS) estimation is essential for predicting concurrent bed. Studies available lacked raw data to facilitate the creation of realistic LoS distributions for dynamic modelling purposes 11–16 To overcome this, initial iterations employed triangular distributions informed by available central tendency data and local expertise in critical care.16
The large standard deviation present in triangular distributions reduces accuracy. Ongoing conversations with frontline clinicians revealed that our model was underestimating the volume of patients experiencing a LoS <48 hours. We were also made aware of a small number of patients tending towards LoS >15 days in both critical and non-critical care. Despite the accuracy of the triangular distribution in initial critical care estimates, we felt modification was necessary to make the model more useful as the pandemic progressed. ‘Best-fit’ analysis of local data on LoS in the first 3 weeks identified distribution curves with a greater rightward skew than initially estimated and sample curves were updated accordingly for patients who did and did not receive critical care. Within the critical care population, separate curves were created for survivors and non-survivors, again, based on the emerging national data.17 Distributions used in the final model are detailed in supplementary material S1.
Model verification and validation
Initial verification of our mathematical model was performed using comparison with international evidence and of the DES model through subject matter expertise. Both models were continually re-validated using local data of inpatient activity and weekly attendances to the emergency department and community hubs.
Outcome
The organisation required weekly predictions of maximum resource use. Our range of predicted results are compared with reported activity (Fig 3). The DES model was able to accurately predict critical care activity until a few weeks after peak infection subsided (week 8). We also predicted that peak critical care activity would start in early April and last 14–21 days. Both of these predictions provided contradictory advice to that issued by the Scottish Government who advised a longer duration peak commencing mid-May (Scottish Government correspondence).
There are several possible explanations for why our non-critical care predictions were less precise and our critical care predictions deviated once peak community activity declined; these are not mutually exclusive. As the initial community response to lockdown was positive, we chose a prediction curve with a rapid incline and rapid decline in new cases. Although timing of peak activity was precise (6 weeks earlier than central estimates), transmission of infection in non-critical patients did not fall as rapidly as predicted. Intra-hospital transmission of infection was not within our conceptual model and likely to have increased the volume of hospital COVID positive activity seen in the non-critical care beds despite local data showing reductions in both community presentations and new hospitalisations. Secondly, the ability to discharge patients back to the community proved more difficult than reflected in our initial LoS predictions. The use of long-term care facilities to expedite early discharge ceased during the modelled period and patients remained in non-critical care beds for longer than organisational leaders expected. Finally, the national strategy to test all inpatients over 70 years old every 3 days (introduced in late April) is likely to have increased the volume of asymptomatic cases detected, increasing hospital activity but not community cases.
Conclusion
Centrally issued predictions early in the pandemic provided NHS organisations with recommendations to generate four to five times their standard critical care capacity due to huge uncertainty. The breadth of these initial predictions is understandable as they used early information and incomplete data, but failure to revisit them through revalidation as data emerged rendered them invalid by late March 2020. The national models lacked context and failed to account for the dynamics of healthcare systems rendering them ill-placed to advise local organisations.19 In contrast, contextualising our model, analysing local activity and following international trends led our team towards accurate estimates in time for the organisation to be proactive and mitigate unnecessary resource waste.
We were forced to navigate methodological challenges in an emergency event on a massive scale and were facing a novel infection with no available expertise to rely on beyond influenza pandemic modelling. Huge uncertainties in disease, population and health systems behaviours left us in the necessary position of revisiting our model and predictions on an almost daily basis. Immediate information was vital, as such, our ability to deliver perfection in the form of robust sensitivity analyses was limited. Instead, we adopted a pragmatic approach to develop the most useful model for our organisation. We whole-heartedly acknowledge the inelegance of the methodology presented and are currently exploring how this could be approached differently for future applications.
While early epidemiological models of pandemic activity are vital to set the scene for national policy and planning, they provide limited usefulness for healthcare organisations charged with managing resources at the local level. We have shown that close collaboration with expertise in DES modelling is a valuable way for organisations to harness some of the uncertainty experienced. The value in this type of research using computer simulation modelling in healthcare is gaining appreciation in academia, but we hope our case study will encourage healthcare teams to explore its use in future health systems planning for both elective and urgent care.
Supplementary material
Additional supplementary material may be found in the online version of this article at www.rcpjournals.org/fhj:
S1 – Discrete event simulation model entities and assumptions.
- © Royal College of Physicians 2021. All rights reserved.
References
- ↵
- Ferguson N
- ↵
- The Scottish Public Health Observatory
- ↵
- Deming WE
- ↵
- Roser M
- ↵
- European Centre for Disease Prevention and Control
- ↵
- Ministero della Salute
- ↵
- Mizumoto K
- ↵
- Day M
- ↵
- Scottish Government
- ↵
- Public Health Scotland
- ↵
- ↵
- Zhang G
- ↵
- ↵
- ↵
- Yang P
- ↵
- Law AM
- ↵
- Intensive Care National Audit and Research
- Wynants L
Article Tools
Citation Manager Formats
Jump to section
Related Articles
Cited By...
- No citing articles found.