Using data to drive service improvement: false dawns and a promised land?

Are current data sources sufficient to support our data aspirations?

The importance of answering this fundamental question is supported by the findings of various NHS-commissioned reports and reviews undertaken in the recent past, including the Wade-Gery report⁷ and Goldacre review,⁸ and will need to be covered by the recently initiated inquiry into NHS Digital Transformation.⁹

While these formally commissioned reports and reviews contribute to a better understanding of the high-level difficulties faced in constructing a system to support the NHS, including the development of quality improvement (QI) methodologies, calls to look at the utility of the basic data sources and for the NHS to develop an all-encompassing digital plan seem to gain little traction.¹⁰

A full exposition of the difficulties of creating a sustainable system to support NHS requirements is beyond the scope of this paper. But the gap between expectation and current delivery is highlighted by considering several common themes in the recommendations of many GIRFT reports:

• A closer working relationship between clinicians and clinical codes: this would improve the accuracy of coding and, as part of the feedback loop, improve the way medical records are presented to coders. This would facilitate clearly presented information.

• More timely access to data sources such as Hospital Episode Statistics (HES): this would enable metrics that are dependent on the data sources to be more frequently updated and more relevant.

• More regular updating of coding systems to keep pace with clinical practice: this is particularly the case with the OPCS Classification of Interventions and Procedures with a call to shorten its revision cycle of three years.

• Improved access to resources held both within and outside the NHS on various databases and clinical registries: without such access metrics cannot be meaningfully developed in many more complex specialties. For the earlier GIRFT reports. HES proved to be sufficient to develop reliable metrics, but as reports addressed more diverse specialties, the development of metrics became more difficult and for some impossible. This applies particularly to complex medical specialties and support services, such as pathology.

In some ways this realisation of the limitations of routinely collected data sources parallels the situation which led to the creation of many clinical registries. For complex specialties, the use of HES data and other mandated data collections did not, and does not, allow the true nature of the clinical reality to be reflected, so it was concluded by clinical groups that a separate parallel data collection system was necessary to cover their area of interest.

Since 1974, over 100 registries have been created. This shows the desire of clinical teams to understand the clinical picture nationally and to develop metrics and reporting to allow accurate audit, service monitoring and QI.

The inability of mandated NHS data flows to fulfil these requirements is not a surprise as the Commissioning Data Set (CDS) design, now in version 6.4, is intended to reflect activity, costing and commissioning, not to reveal the true complexities of clinical practice.

The closest the NHS gets to reflecting clinical practice is the Health Resource Group (HRG) design by the Expert Working Groups (EWG) at the National Casemix Office (NCO) within NHS Digital.¹¹ These look at HES and costing data to design clinical groupings which reflect as best they can current practice within the limitations of ICDv10 and OPCSv4. The current HRG4+ design has about 2.5K HRGs across 34 EWGs and has been used as the basis of activity reporting and payment for 20 years.

However, clinical registries focus on the clinical data and are far more responsive to changes in clinical practice, especially with specialised commissioned services, many of which are reflected in the development of specific quality dashboards¹² as part of the Quality Improvement Programme (QIP) run by NHS England.

Although laudable and understandable, the development of a wide range of clinical registries has been carried out in an unplanned and highly variable way. Each registry has a very individual history but, having been co-developed by clinical teams with software engineers, they reflect clinical practice accurately and enjoy high levels of commitment by clinical teams to data acquisition. The data are typically both complete and highly granular. In many instances they maintain higher data quality and case ascertainment than NHS-mandated data flows. However, many registries have limited follow-up data and would have their capabilities enhanced with routine linkage to other health datasets and to mortality data.

The principal difficulty with registry data is that they are usually held outside of the NHS in such institutions as universities, specialty associations and charities; thus the governance challenges of bringing such data into the NHS are significant, although not unsurmountable.

However, bringing clinical registry data into the NHS is key to the development of metrics for quality assurance (QA) and improvement purposes for the more complex clinical services.

Data linkage

A review of registries by NHS England with the intention of assessing their utility has been attempted on three occasions since 2013 but no conclusions or recommended actions have arisen from these reviews. Part of the problem was the huge variation between the registries. No single plan would fit all.

This importance of this issue can be illustrated by considering the MDSP, as it requires the scrutiny of linked NHS data over the long term to identify the occurrence of a clinical complication which may be rare. For the MDSP to work, identifiable patient-level clinical information needs to be recorded alongside the unique device identifier (UDI) of any implanted device. As the trend is towards shorter lengths of stay and reduced secondary care follow-up, this means that any device-related impact on health may only appear in primary care or community care and their related data systems. Any related cancer or mortality will be apparent in the National Cancer Registration and Analysis Service (NCRAS) or the Office of National Statistics (ONS) mortality database.

Analysis of these combined data sources in the long term will be necessary, and sophisticated statistical techniques will be needed to identify adverse health effects which may be attributed directly to a device rather than patient comorbidities or other extraneous factors. Data linkage and appropriate methodology are essential prerequisites for the successful development of an early warning system for health-related associations with an implanted device.

A not dissimilar pattern of data linkage and scrutiny is required to identify outlier clinical practice within a service or by an individual or team of individuals. This will also require a means of analysing data on all clinical work undertaken in the independent sector as well as the NHS and would ideally cover all devolved nations of the UK.

Data quality

All this depends on data quality (DQ) and ascertainment. Currently very little attention is given to DQ from providers, although specific clinical registries have worked hard through feedback mechanisms to local teams to drive this important issue. The Data Quality Maturity Index (DQMI) was created by NHS Digital in 2018 and measures the provision of a small number of key data items in data from NHS providers.¹³

For most data systems in the NHS, there is no formal feedback to providers and certainly no surveillance with checks and responses by the NHS to maintain DQ. It is all left to the providers, including those in the independent sector undertaking NHS activity, to monitor their own DQ behaviour.

Recent changes to the payment system for most of the NHS using the aligned payment and incentive system¹⁴ have given rise to concerns that DQ will suffer from 2022/23 onwards as the pressure provided by PbR on providers to maintain DQ will be progressively lost. These and other concerns have been formally expressed by the combined EWGs.¹⁵

How can we improve matters?

One finding by the inquiry into NHS data transformation is easily predictable: that current data systems rarely provide frontline staff with what they need. Solutions are imposed rather than co-developed and frequently add to the administrative burden. This runs the danger of disenfranchising the clinical community, with the result that DQ and completeness deteriorates across the board.

The health secretary's desire for all providers to have an electronic patient record (EPR) in place by the end of 202316 partially misses the point. Making records digital within a single organisation is helpful, but it is merely a precursor to the availability of linked clinical data to all clinical staff working across a health economy or nationally, which is key to improved clinical delivery.

It is at the frontline that the best data are available. In fact, the aspiration must be to collect data as a function of creating and updating clinical records, avoiding the ‘once removed‘ problem that affects most NHS data collection currently. This includes clinical coding and registry completion, which are not carried out in real-time and are one step away from where clinical care happens. Using an EPR to allow coding in the background, undertaken directly from an appropriately structured clinical record and without requiring clinicians to carry out direct coding, was the concept behind SNOMED CT and applies also to ICD¹¹, should either of these systems be fully introduced.

The continued design and use of data systems over and above direct record keeping will inevitably add to the administrative burden of staff and further pressurise the NHS. The impact on the workforce is inevitable while they struggle with the lack of information support required to improve the care of patients.

Extending this argument one further step suggests that the artificial separation of data collection, analysis and reporting between commissioning, QA and QI is unsustainable.

The current systems and processes were often created to meet specific short-term problems and the overall system has evolved rather than being designed to produce specific outputs. The silo thinking this approach encouraged has led to overlapping functions between systems with no overall comprehension or cohesion. Perhaps the systems are perceived as too complex for this. That is not the case.

What we have now is a fragmented system because these short-term solutions persist and seem to be added to constantly, increasing the waste of time, money and effort as a consequence of the duplication in collection, analysis and reporting. It is arguable that the current situation is not fit for purpose and certainly not one which will meet the anticipated needs and aspirations of the NHS.

The recently published tender¹⁷ for the creation of a federated data platform (FDP) describes a potential way to resolve some of the difficulties described in the various GIRFT reports and support the aspirations of the entire NHS described in the NHS Long-term Plan (LTP)¹⁸ and the Health and Care Act 2022.¹⁹ The plan appears to include all data sources: community, primary care, mental health, secondary care and others such as NHS databases and clinical registries.

However, the FDP plans do not appear to acknowledge the issues with DQ and data utility that currently exist with these data sources. The existence of these problems is appreciated by some but the work and infrastructure required to create a functional programme may not be fully understood by senior management. Work on integrating systems should focus as much on DQ issues as on the technical infrastructure; this will require a strong interface with clinical expertise to ensure appropriate data are collected, properly analysed and understood.

It is technically possible for the FDP to provide all functions required by the NHS:

• Activity and patient access monitoring based on population-based commissioning principles to assess the suitability of provision, taking into account travel times and levels of population deprivation.

• Demand and capacity estimations using methodologies such as statistical process control to quantify variations in demand over time and calculate the required capacity for a current or planned change to a service.

• Linked patient-level costing data to allow the financial consequences to be evaluated.

These approaches are, of course, the basic components of intelligent health commissioning at whatever level it is organised and would enable a value-aligned approach to commissioning that would ease local, regional and national decisions on service provision; provide an integrated system for QA, QI and appropriate monitoring of patient safety, plus evaluation of the patients' experiences; and aid and enhance clinical research.

Ideally, centralised clinical and analysis expertise and resource would be used to create an NHS-facing secure clinical data environment (CDE). Associated with this would be the creation of reliable metrics as seen in the QIP, GIRFT, NCIP and several clinical registries to act not only as QA and QI tools but also as a clinical early warning systems for the NHS. Such metrics from linked data would also allow peer service comparison and support individual clinical appraisal.

Creating a CDE with the necessary levels of linkage designed into the system would answer some of the key operational questions as well as providing the necessary data blueprint for a trusted research environment (TRE). A suitable anonymisation methodology and a carefully controlled access process to the TRE would greatly ease access to the linked data for approved research.

The challenge of controlling access sufficiently to comply with the requirements of the Information Commissioner's Office (ICO) and relevant data governance legislation while still providing a ‘single source of the truth‘ for the NHS is significant, but, again, not insurmountable if the political will exists.

Finally, this should be a UK-wide rather than England-wide system, in recognition that people travel for their care.

Steps along the way

In the first instance, the mandated commissioning datasets must be reviewed to reflect QA and QI aspirations where they can, while registry data must be brought in to fill the immediate gaps. Later, careful consideration can be given to replacing some registry collections with an improved CDE and, where this is not possible, the registries themselves need to be brought under the NHS data umbrella to allow near real-time data access for linkage and reporting. The system should be designed to be constantly adaptive to accommodate changes in clinical practice.

Admittedly, previous attempts to develop integrated systems in the NHS have been very expensive and ultimately unsuccessful. It is to be hoped that the FDP will be better if due notice is taken of the existing problems.

In the short term, it must be appreciated that the high-level aspiration for a fully integrated system for monitoring and improving clinical care is still a long way off and there is no amount of inventive data analysis or business intelligence reporting that can polish the existing data sources into a fit state for the sophisticated and wide-ranging expectations currently being discussed.

There is a risk that inventive reporting from flawed systems will, by its very mode of presentation, be accepted as the truth. This is hardly without precedent in the NHS and beyond. The risks of this form of health science disinformation must be avoided. The first step would be to recognise how much of it already exists.

This growing risk can be mitigated by employing the slowly growing number of clinical informaticists within the NHS who understand the interplay and co-dependence of clinical knowledge, analytical methodology and technological capabilities and possess skills in all three areas. There is a call for such individuals to be permanent members of trust boards, integrated care boards (ICBs) and, especially, the NHS Board. Clinical informaticists can draw on specific clinical expertise when required. EWGs are an ideal access point, as are the clinical leads of registries.

There is a legitimate concern that the future use of healthcare data in the UK will not fulfil its potential if the current well-recognised shortcomings of existing systems and processes are not dealt with. Our collective role is to see that this is not allowed to happen. Perhaps the first task is to push for an improvement in the monitoring and response to any DQ decay.

Being pragmatic, the size and daily operational importance of the entire data system means fundamental re-design is not possible. Development will have to be incremental, based on a plan which acknowledges the problems which exist. However, as yet, no such plan exists. It will need to be long term, well beyond the term of a government, and with the necessary resources to make it happen. The ₤240 million commitment to the FDP is a start.

The role of data is to allow better understanding of needs, to plan service delivery, measure the quality of that service delivery and to allow the additional benefit of supporting much-needed research. The primary aim must be to support frontline healthcare delivery to bring about improvements in outcomes and patient experience based on what is found.