Improving quality of care through national clinical audit
ABSTRACT
The UK's national health services are unique in having a well established programme of national clinical audits and databases across medical, surgical and mental health conditions. The Royal College of Physicians’ (RCP) Clinical Effectivess and Evaluation Unit leads many of the largest and most mature audits in partnership with specialist societies, other colleges, patient groups and academic institutions. In this paper, we shall trace the development of national audit over the last 2 decades, explore the mechanisms by which this has helped improve care and discuss how national clinical audits and databases can best support quality improvement in the NHS of the future.
Introduction
Since 1998, when the Royal College of Physicians (RCP) launched the first national clinical audits,1 the centrally-funded programme in the UK – led by the Healthcare Quality Improvement Partnership – has grown to include 28 audits and databases covering the whole range of medical, surgical and mental health conditions.2 Alongside this expansion, national audit methodology has matured from snapshot, anonymous data collection on paper pro formas with reporting some months later to continuous, prospective online data collection and near-real time reporting.
The RCP's Clinical Effectiveness and Evaluation Unit now collects prospective data on almost every patient in England with stroke, hip fracture and lung cancer; from 2017, the national COPD Audit will implement this for patients admitted to hospital. National audit data are freely available to patients and the public on the internet, and improvements in methodology and information technology have meant that outputs are robust enough to support high-quality health services research, clinical guideline development and use by regulators, policymakers and commissioners.
Increasing sophistication of audits and databases has been accompanied by very significant improvements in the care of patients and, although not entirely attributable to the audits, the provision of high-quality comparative data has been a real stimulus for improvement. In this paper, we will examine the mechanisms by which this improvement has occurred, with a focus on those audits led by the RCP.
Approaches to improving quality in healthcare
Numerous approaches have been advocated to improve the quality of healthcare and all have their strengths and weaknesses (Table 1).
They can be broadly divided into those which rely on a ‘top-down’ influence on those who lead or finance systems (eg national targets and directives or using the payment mechanism) and ‘bottom-up’ ones which rely on the intrinsic professional motivation of clinicians and staff (eg clinical audit, local quality improvement initiatives).
‘Top-down’ improvement
The NHS has traditionally relied heavily on central directives and policy initiatives to improve quality. Advocates of this approach point to reductions in hospital associated infections following introduction of national targets as examples of its success, but international experts have warned that the NHS overrelies on ‘hierarchy, inspection and markets’.3 Professor Don Berwick and others, in a King's Fund report, describe ‘a bewildering and overwhelming profusion of government-imposed policies and programmes’.4
The NHS in England has also enthusiastically embraced the use of the payment system to incentivise good quality. However, evidence from the USA is that enhanced payments for achieving quality targets or withholding of payment for not doing so, has had largely disappointing effects on quality.5 A review of international experience found some common themes: policymakers tend to overestimate the benefits of payment incentives, they make requirements for achieving these too complex, and they underestimate the administrative burden and complexity of introducing them.6 Providers also tend to focus on achieving, but not necessarily exceeding, the targets to trigger payments and there is a certain amount of ‘gaming’ in all such systems.
Advocates of public reporting of healthcare outcomes have promoted it as a mechanism to improve quality on the basis that, faced with information on outcomes of local providers, patients would function as consumers in a market and choose the highest quality providers. The evidence does not support the view that a market in healthcare operates like this.7 In fact, patients are much more likely to choose health providers based on recommendations from family and friends, convenience and previous personal experience. Public reporting may improve outcomes through other mechanisms though, including enhancing competition between providers.
‘Bottom-up’ approaches
Clinical audit began as a professional initiative to allow clinicians to compare the care that they were delivering against evidence-based best practice. Although no mechanism for quality improvement was made explicit in the methodology, it was assumed that the intrinsic professional motivation of clinicians would lead them to work to improve once they were aware of deficiencies in care. Other professionally led initiatives, like accreditation schemes and confidential enquiries, also rely heavily on awareness-raising to stimulate improvement.
National audits were developed by clinicians working in specific clinical areas, often as a response to recognition that the services provided were of inadequate or variable quality (eg stroke, heart attack, hip fracture). Early national audits were criticised for lack of methodological robustness, for the burden that data collection imposed on front line staff and for the focus on data collection without being linked to recognised quality improvement techniques. There were also perceptions that national audits were designed to be relevant only to doctors, not to health service policymakers, managers or other clinicians. Although most national audits remain largely focused on secondary care, they are increasingly moving to include community and primary healthcare. The National COPD Audit and Sentinel Stroke National Audit Programme assess care quality in pulmonary and post-stroke rehabilitation settings, respectively.
‘Industrial’ approaches to quality improvement (QI), like the Model for Improvement, Lean, Toyota Production System and Six Sigma, were introduced into healthcare by the Institute for Healthcare Improvement (IHI) in the early 2000s and promoted in the UK by NHS improvement bodies.8,9 These approaches rely on training front line staff in practical QI methodology on the understanding that they are best placed to understand and therefore fix the problems that their patients experienced. However, results from large scale QI projects based on this methodology in both the UK and USA have had disappointing results,10–13 with many projects reporting poor clinician (especially doctor) engagement, lack of alignment with other operational initiatives and lack of coordinated leadership. The failure of many of these programmes to deliver the hoped for results may not be the fault of the QI methods themselves, but more a poor understanding of how to use them effectively in practice.14
Have national clinical audits and databases improved care?
Stroke care has improved unrecognisably since the RCP stroke programme was established in 1995. The national clinical audit was led by a broad coalition of clinical partners1 and helped inform a range of policy initiatives prioritising stroke in the NHS over the next decades. These involved a national stroke strategy, the use of audit data by regulators and commissioners, publication of data for the general public and the introduction of payment incentives for good stroke care. The NHS invested heavily in stroke services and stroke medicine grew as a specialty, which was reflected in reductions in length of stay, mortality and proportion of patients discharged to nursing homes. Since 2010, prospective data on all stroke patients in England, Wales and Northern Ireland have been included in the audit. With such a large database, it is now possible to analyse data to understand issues like how staffing patterns influence quality of care or how temporal varations in quality arise across the week.15–17
As happened with stroke, the National Hip Fracture Database (NHFD) was established by a small group of enthusiastic clinicians who were concerned about the poor quality of hip fracture care in the UK.18 Data collection began in 2007 and it has now become the largest hip fracture database in the world, with records of 450,000 patients – including data on every patient with a hip fracture in England, Wales and Northern Ireland. Since 2007, hip fracture mortality at 30 days has reduced from around 11% to 7.5%, and there have been significant improvements in many other process and outcome measures. As in stroke, the development of the hip fracture database coincided with a focus on hip fracture by NHS policymakers and professional bodies. The introduction of a payment incentive linked to achievements in the audit in England was followed by a sharp improvement in a range of process and outcome measures.
An unpublished RCP audit of lung cancer treatment in 1995 showed long waiting times and wide variations in access to treatment and was one of the stimuli for the development of a national lung cancer audit.19 As in stroke and hip fracture, the development of the audit was accompanied by various policy and other initiatives to improve care. The national audit now collects prospective data on almost every lung cancer patient in England and Wales and at a national level there have been significant improvements in survival, with a reduction in delays to treatments. However, even with robust case mix adjustment, the data still show wide variability in quality of care across the country, a pattern that is evident in stroke and hip fracture care as well.
The RCP leads national clinical audits in several other conditions and specialties (COPD, inflammatory bowel disease, end-of-life care etc) and is a partner to other professional bodies in others. All are at different stages of maturity, but almost all have been accompanied by significant improvements in the care of patients with the relevant condition.
What are the mechanisms by which care has improved?
There is no one clear mechanism by which care has improved with national audits. For many conditions, the development of a national audit has occurred in parallel with NHS policy changes and increasing professional awareness of quality gaps.
A component of improvement has undoubtedly related to awareness-raising. Robust evidence that quality is variable or falls below evidence-based standards tends to bring a focus from professional bodies, clinicians, NHS managers, policymakers, patient advocacy groups, the press and the general public. This has often led to increased investment in what were previously neglected clinical areas (like hip fracture, stroke or end-of-life care). During the time that many national audits were being developed, there was a very significant increase in NHS spending so additional investments were not only seen as justified but also affordable. For many conditions, audit programmes developed in parallel with NHS policy initiatives led and supported by groups of enthusiastic clinicians and patient groups. Professional leadership and prioritisation tended to engage clinicians, and policy initiatives aligned this with the priorities of non-clincial managers.
NHS leaders and policymakers also tend to pay particular attention to issues raised in the press. In light of this, the RCP audit programme has developed an increasingly sophisticated approach to the press in recent years, taking a proactive approach to press conferences and press releases. Although it is not always possible to prevent mis-reporting, media attention has been a helpful stimulus to support RCP work in areas such as asthma and end-of-life care.20,21
Patient groups have also had an important role in challenging poor quality care; for example, Age UK, Asthma UK, the British Lung Foundation, Crohn's and Colitis UK, Marie Curie, Roy Castle Lung Cancer Foundation22 and the Stroke Association have all been important advocates for the RCP's national audits. Patient advocacy works at a local as well as a national level and the RCP has developed its approach to public reporting accordingly. We have produced specific publications for patients based on national audit outputs, where data are framed to inform patients and carers as to what to expect from their local services (eg My hip fracture care and Hospital care in the last hours or days of life).23,24
Data from national audits are also available for NHS regulators and for those who commission care; some audits, such as stroke, provide data specifically by clinical commissioning group in England and by palimentary constituencies to enable members of parliament to understand how their local services are performing.25 Peer review publications based on the secondary use of audit data have made important contributions to wider aspects of public health, including the provision of robust data on the variability of quality by time of day or day of the week and the relationship between quality outcomes and staffing levels.15,16
Some audits, notably the NHFD, have achieved success through linkage with payment system incentives and several others have established or explored similar approaches.18 Many clinicians are enthusiastic about this approach, although in times of shrinking NHS resources it is not clear that widespread linkage of national audit to payment incentives is feasible. There have also been examples of less successful attempts to use the payment system. Up until 2014, the national end-of-life care audit was based on the Liverpool Care Pathway (LCP), the use of which was also a national policy priority. While there were multiple factors that led to the LCP's abandoment, the linkage of its use with a financial incentive to place dying patients on the LCP may have led to its inappropriate use and was certainly widely misrepresented by the media.26
Many audits have developed linked QI initiatives. A stroke peer review scheme is offered by the RCP and its partners to those seeking to improve services27 and a similar scheme supporting the NHFD is run jointly by the British Orthopaedic Association and the British Geriatrics Society.28 An ethnographic evaluation of a reciprocal peer-to-peer review programme linked to the national lung cancer audit demonstrated that such an approach could be a powerful mechanism by which to improve quality.29,30
Accreditation schemes have also been linked with national audit to improve quality. As a result of early audits of care for patients with dementia, the Royal College of Psychiatrists developed the Elder Friendly Quality Mark for general hospital wards, containing a set of standards that are required in order to achieve accreditation.31
The RCP has also led two Health Foundation funded QI programmes based on the IHI Model for Improvement in inflammatory bowel disease and in falls, which has led to the development of the Fallsafe care bundle and Carefall training packages.32,33
What are the components of a successful national audit?
Professional clinical leadership and the existence of robust, evidence-based guidelines are pre-requisites for successful audit. The clinical condition being audited must be sufficiently common, and there must be sufficient evidence of deficiencies in care to justify the extensive infrastructure and investment required. Clinical enthusiasm for a national audit must be matched by that of patient groups, policymakers and NHS leadership. Participation is enhanced if clinicians clearly associate themselves with the lead professional body but may be undermined if professional bodies are not aligned or if they seem to have conflicting approaches. Regular feedback to and contact with local clinical teams helps to maintain engagement.1,18,19
It is helpful if there are existing well-validated process and outcome measures of quality and if data can be readily collected by busy clinicians or is routinely generated through clinical encounters. Datasets need to be as small as possible in order to achieve the primary aim of improving care. The methodology needs to be robust enough to provide data for regulators, commissioners, health services researchers and the public. Linkage of national audits to recognised quality improvement tools allows local clinical teams to work with their own data to improve care. At the same time, well designed national audits can contribute to decreasing the overall burden of data collection placed on clinicians and organisations by replacing multiple local and national data collections with a single, high-quality and nationally benchmarked source of data.
Because of the considerable resource and infrastructure requirements, national audits and databases are not likely to be suitable for every circumstance. There have been particular challenges extending audits into primary care or to cover care before or after hospitalisation. This has been partly related to technological challenges (linking information from different datasets) and partly due to changing and increasingly stringent rules on data sharing and information governance. In addition, there is less clarity about optimum models of care outside hospital and less of a tradition of national audit in primary and community care.
Conclusions
National clinical audits and databases have made, and continue to make, a real difference to the care of NHS patients. The most successful ones have strong professional leadership and are linked with a range of approaches to improve quality, harnassing intrinsic professional motivation within an environment in the NHS that supports improvement through alignment with policy and other initiatives. Sound scientific methodology and the ability to use data for a range of purposes help audits grow and develop.
In the future, audits and databases will need to continue developing to meet the challenges of different care models and will have to support improvements in care in an NHS with ever increasing pressures on funding. There are many opportuntities to make national audit more powerful, responsive and efficient: smaller and more standardised datasets, making use of new digital technologies, improved quality of routinely collected data, more advanced use of analytics and data visualisation, and building in continuous quality improvement methods. National audits and databases have evolved hugely over recent years, given us some of the most powerful insights into the quality of healthcare and how it can be improved, and will be central to future data-aware health services built on a culture of continuous quality improvement.
Conflicts of interest
The authors have no conflicts of interest to declare.
- © Royal College of Physicians 2016. All rights reserved.
References
- ↵
- Cloud G
- ↵
- Healthcare Quality Improvement Partnership
- ↵
- Ham C
- ↵
- Ham C
- ↵
- James J
- ↵
- ↵
- Marshall M
- ↵
- ↵
- The Health Foundation
- ↵
- Benning A
- ↵
- ↵
- ↵
- Vincent C
- ↵
- Reed JE
- ↵
- ↵
- ↵
- Stewart K
- ↵
- Boulton C
- ↵
- Beckett P
- ↵
- Hughes D
- ↵
- Mundasad S
- ↵
- Roy Castle Lung Cancer Foundation
- ↵
- Royal College of Physicians
- ↵
- Royal College of Physicians
- ↵
- Royal College of Physicians
- ↵
- Doughty S
- ↵
- Royal College of Physicians
- ↵
- Gray AC
- ↵
- ↵
- Aveling E
- ↵
- Royal College of Psychiatrists
- ↵
- Royal College of Physicians
- ↵
- Royal College of Physicians
Article Tools
Citation Manager Formats
Jump to section
Related Articles
- No related articles found.