Exploring variations in lung cancer care across the UK–the ‘story so far’ for the National Lung Cancer Audit
Abstract
The National Lung Cancer Audit was developed to improve the quality and outcomes of services for patients with lung cancer, knowing that outcomes vary widely across the UK and are poor compared to other western countries. After five years the audit is capturing approximately 100% of the expected number of incident cases across hospitals in England, Wales, Scotland, Northern Ireland and Jersey. Measures of process and outcome have improved over the audit period, such as the histological confirmation rate (64–76%), the proportion of patients discussed in a multidisciplinary team meeting (78–94%), and the proportion of patients having anti-cancer treatment (43–59%), surgical resection (9–14%) and small cell lung cancer chemotherapy (58–66%). These national averages hide wide variations between hospitals providing lung cancer care which cannot be accounted for by differences in casemix. This paper describes the evolution of the audit, and describes the ways in which it may have improved clinical practice.
Introduction
Lung cancer is the most common cause of cancer death in the world, with over 40,000 new cases and over 35,000 deaths per annum in the UK.1 Malignant mesothelioma of the pleura, a separate and uncommon thoracic cancer is increasing in incidence as reflecting historic occupational asbestos exposure. Data obtained from the Office for National Statistics in the early 1990s showed a fourfold difference in five-year survival rates between health authorities in England and an unpublished audit run by the Royal College of Physicians (RCP) in 1995 confirmed long waiting times for treatment and wide variations in resection rates.2 Subsequent studies have described differences in management and survival between the UK and other comparable European countries.3,4 It has always been unclear to what extent such variation could be explained by casemix variables (ie patients diagnosed later with more advanced disease or multiple co-morbidities) and to what extent they are the result of differences in standards of specialist care, highlighting the need to establish a high quality national lung cancer audit (NLCA), with the aim of recording information about activity, process and outcomes in lung cancer and, using casemix adjustment, begin to explain the wide variations in outcome. Although the UK cancer registries have collected data on lung cancer since the 1970s they have limited information on a number of important factors, including treatment and the key casemix variables, particularly stage. While elements of this information are contained in Department of Health (DH) hospital episode statistics (HES), and the similar patient episode database Wales, these sources do not reliably collect clinical data at the level of detail needed for clinically relevant national comparison.
Development of the audit
A core dataset was published in 1999 following a series of multidisciplinary workshops, with the audit programme being commissioned and funded in 2004 by the DH National Clinical Audit Patient Outcomes Programme.5 The management of the audit was taken on by the NHS Information Centre under its National Clinical Audit Support Programme in collaboration with the Clinical Effectiveness and Evaluation Unit (CEEU) of the RCP, leading to the development of all the detailed supporting documentation and methodology, including a central information technology solution to collect, house and analyse the data. The management structure consists of an overarching project board alongside a project team consisting of clinical leads, project management and support staff. A clinical reference group of experts in various specialty areas and including patient/carer representation provides clinical oversight. In June 2007, a new version of the dataset was adopted to reflect changes in practice and the publication of national guidelines since the original dataset had been developed (for example positron emission tomography scanning and specialist nursing input).6 The current dataset consists of a maximum of 112 fields covering demographics, referral patterns, investigations, disease stage, co-morbidity, treatment and outcome.7 The dataset (appendix 1–available online only) focuses on lung cancer but also collects data on malignant pleural mesothelioma.
An online data collection tool (the LUCADA database) was developed to collect this core dataset from hospitals in England, allowing either direct data entry on individual patients, or upload of data from local systems using CSV or XML files (appendix 2 – available online only). A users group was set up to discuss issues with this tool and contribute to its development. A telephone helpdesk is provided by NHS Connecting for Health. Following a short period of data collection from four pilot sites, a programme of national rollout began in June 2004. The different organisation of healthcare in the different countries of the UK makes a consistent approach to data collection very difficult. However, from 2005, data from Welsh trusts, collected through the use of their Cancer Network Information System Wales database, was included in the audit. Since 2008, Scottish trusts submit a subset of their own data in a format comparable to that for Wales and England, though their approach has not yet permitted the use of case-mix adjustment. Data from Northern Ireland and Jersey were submitted in 2010, but are not mentioned further in this paper. Therefore, unless otherwise stated, the results quoted in this paper refer to England and Wales only. Data are collected in different ways in different organisations–sometimes by clinical staff and sometimes by data clerks, but recording of key clinical data at the lung cancer multidisciplinary team meetings has always been encouraged, as this is the forum where such data are used to drive clinical decisions.
A deadline of 30 June is set for the upload of the previous calendar year's patients' data prior to analysis. Results are analysed centrally based on the ‘date first seen’ and ‘place first seen’ for patients seen by a specialist in secondary care, and are generally reported by NHS trust and by cancer network (geographical groupings of several trusts adopting the same pathways and guidelines). Statistical analysis of the data has been carried out by different teams at different stages of the project, but has generally utilised typical statistical packages such as SPSS or Stata.
Casemix adjustment
In order to correct for differences in casemix, logistic regression models were fitted to the data and casemix-adjusted odds ratios calculated to compare the odds of a treatment or outcome occurring in one organisation compared to a baseline, making adjustments for sex, age, stage, performance status (PS) and deprivation. An odds ratio greater than one indicates that the odds of treat ment or outcome occurring were higher in the comparator rather than the baseline, and vice versa. Not all possible factors that may affect treatment rates could be adjusted for in these models due to lack of data (eg co-morbidities). For analysis of mortality, a similar process of case-mix adjustment is carried out to produce hazard ratios which are interpreted in a similar way, with a ratio greater than one indicating a higher risk of mortality.
Results
Case ascertainment and data quality
Support for the audit has grown steadily and now all hospitals managing lung cancer in the UK are participating. The number of cases submitted has risen from approximately 10,000 for England only in 2005, to 37,298 for England, Wales, Scotland, Northern Ireland and Jersey combined in 2009. This latter figure represents around 97% of the annual incidence of cases of lung cancer and around 100% of those presenting to secondary care (based on historical cancer registry incidence data). Preliminary data for England only for the 2009 cohort show a further increase with 30,155 cases submitted. Overall, by March 2011 over 140,000 cases have been submitted since the audit's inception.
Analysis of the data from England and Wales for the most recent cohort (2009) demonstrates a male to female ratio of 1:1.4. A further breakdown by sex, age and cell type is shown in Table 1. Casemix variables of age (derived from date of birth), sex, deprivation index (derived from postcode), performance status, disease stage and co-morbidity are important constituents of the dataset.8 Date of birth, sex and postcode are mandatory fields, but the others are not. Recording of stage and performance status has improved from 47% to 82% and from 53% to 79% respectively.
Process of care
Key indicators of the quality of care provided include waiting times, the proportion of patients having histological or cytological confirmation of their cancer (the histological confirmation rate (HCR)), and the proportion of patients discussed in a multidisciplinary team (MDT) meeting. Dates of key events on the pathway are recorded and are still used, for example, as confirmation that a planned treatment has happened, but the separate National Cancer Waiting Times programme provides information about the timeliness of investigation and treatment.
In 2005, the HCR was 64% for England, and for 2009 was 75.6% (interquartile range (IQR) 71–85) for England and Wales combined. The proportion of patients discussed in an MDT rose from 78% in 2005 (England only) to 94% in 2009 (England and Wales). However, these averages hide wide variations across the cancer networks (64–86% for HCR and 79–99% for MDT discussion) and as would be expected, the variation by trust is even more extreme. It is likely that poor quality data are the explanation for the extreme outliers. The HCR might be expected to be sensitive to casemix (less fit patients might not be suitable for invasive biopsy techniques), and Fig 1 shows the casemix-adjusted odds ratios for histological confirmation for all lung cancer across the cancer networks. Further analysis shows a striking correlation (Spearman's correlation co-efficient 0.95, p<0.0001) between the unadjusted and adjusted odds ratios. These findings suggest that casemix does not explain the whole of the variation between networks in the HCR.
Outcomes of care
The audit collects data on treatment rates and survival (from date of diagnosis to death), with the date of death field being linked to data from the DH Personal Demographics Service which is then validated against Office for National Statistics data offline. Overall for 2009, 59.1% of patients had anti-cancer treatment (surgery, chemotherapy or radiotherapy (43% in 2005)), 18.3% of patients with histologically-confirmed non-small cell lung cancer (NSCLC) had some form of surgical operation in an attempt to cure their disease (14% in 2005), and 65.7% of patients with small cell lung cancer (SCLC) had chemotherapy (58% in 2005). It can be appreciated that most measures of outcome have improved during the lifetime of the audit although the relative contributions to this from changes in practice and improvements in data collection are uncertain.
Once again, these averages hide wide variations which persist after casemix adjustment as shown in Fig 2. The correlation between unadjusted and adjusted outcomes are very strong–for anti-cancer treatment in NSCLC (Spearman's 0.74, p<0.0001), for resection rate in NSCLC (Spearman's 0.82, p<0.0001), and for chemotherapy in SCLC (Spearman's 0.85, p<0.0001). Figure 3 demonstrates the casemix adjusted hazard ratios for mortality, based on the median survival data for the 2009 cohort. Once again it is noted that that there is variability across the networks, although the variation is not as extreme as for other measures of process and outcome.
Discussion
While research continues into screening, earlier diagnosis and new anti-cancer treatments, it is vital that the best current standards of care are applied to all patients.9,10 The results from the audit show that national comparative audit in the NHS is feasible, and has demonstrated wide variations between trusts and networks in the quality of care given to lung cancer patients in England and Wales. Although differences in casemix are often used to explain such variations, they are still apparent after adjustment for casemix with very strong correlation between unadjusted and adjusted results. The experience and lessons of the audit are applicable across the wider NHS in an era where measurement of clinical outcomes forms a core principle of its future direction.11
While the data collection has been a success, there are some key lessons that are relevant to future similar projects. One of the main barriers to participation in the early years was the complaint that the data lacked credibility because of its poor data completeness, but this initially widespread viewpoint has been entirely reversed as collection has improved, highlighting the need for persistence and a longer-term view in planning and funding. Likewise, a very detailed dataset capturing all the minutiae of the patient journey has to be balanced against the practicalities of collecting too much information. Regular feedback and reporting of data has been a cornerstone of building confidence and interest in the audit. An annual report is sent to all hospital chief executives and medical directors, primary care trust (PCT) cancer commissioners, MDT and network leads and is available for download from the NHS Information Centre website (www.ic.nhs.uk/services/national-clinical-audit-support-programme-ncasp/cancer/lung). Headline outcome measures are fed back to individual trusts and networks every quarter and numerous presentations and abstracts at local, regional, national and international meetings have been undertaken by the project team. Furthermore, the LUCADA data collection tool produces real-time reports of an individual trusts' submissions to the audit, benchmarked against a national average.
The casemix adjustment models use those factors that are considered to be relevant in these patients. Co-morbidity is likely to be a further important factor to take into account and data on this have been in the dataset from the outset. There are a number of methods of measuring co-morbidity, but for the purposes of the audit, users have been asked to enter data only on those co-morbidities that influence treatment decisions. Unfortunately a survey of users carried out in 2008 confirmed the impression that, in many cases, the submitted data did not strictly adhere to this definition, making these data impossible to reliably interpret. For these reasons co-morbidity data have been excluded from the casemix adjustment. However, since there is a strong link between the frequency/severity of co-morbidities and deprivation at a population level, and noting that adjusting for deprivation does not alter the findings to any great extent, it may be that more detailed data on co-morbidity would have had little impact on the findings.12 Likewise, although the casemix fields of stage and performance status are variably completed by organisation, a separate analysis of the data suggests that concerns about selective reporting are unfounded.13
The audit is not resourced to address issues at a local level and in order to improve the standards of care and thereby the outcomes for patients, it is vital that networks, trusts, MDTs and individual clinicians take on board the relevant audit findings and use them locally to examine any potential shortcomings and drive service improvements. Local action planning toolkits to assist in this process are provided with the NLCA annual report and are available from the website. As the quality of the audit data has improved, it has begun to be used more widely, for example underpinning research proposals, service planning and redesign, as well as being used as a basis for international comparisons.
Future priorities for the audit include securing ongoing funding, developing the dataset to keep up to date with changes in practice (for example the new staging classification, and the increasing use of targeted ‘biological’ agents), linking other sources of data such as Hospital Episode Statistics, radiotherapy treatment records and results from the National Cancer Peer Review Programme, as well as producing more in-depth analysis of the data to try to hone in on the key factors driving variations in measures of process and outcome. With this in mind, the audit team has recently begun a project funded by the Health Foundation, looking at lung cancer MDT working in England. This project aims to identify the reasons for the variation in lung cancer outcomes at trust level and to address them using standard quality improvement methods. For the first time, this work will include measurement of patients' experience. All of this work is vital if high quality care for all patients diagnosed with this devastating disease is to be secured.
Postscript
Since this paper was accepted for publication, a further NLCA annual report has been published, which can be accessed online at www.ic.nhs.uk/services/national-clinical-audit-support-programme-ncasp/cancer/lung.
- © 2012 Royal College of Physicians
References
- ↵ Cancer Research UK. UK Lung Cancer incidence statistics 2009http://info.cancerresearchuk.org/cancerstats/types/lung/incidence./.
- ↵ Department of Health. National performance indicators for the NHS 2000London: DHwww.performance.doh.gov.uk/nhsperformanceindicators/indicators2000.htm
- ↵
- Imperatori A,
- Harrison RN,
- Leitch DN,
- et al
- ↵
- Møller H,
- Linklater KM,
- Robinson D
- ↵ Royal College of Physicians. Lung cancer: a core data set for the measurement of process and outcome in lung cancer management 1999London: RCP
- ↵ National Institute for Clinical Excellence. The diagnosis and treatment of lung cancer 2005London: NICEwww.nice.org.uk/CG24
- ↵
- The Information Centre for Health and Social Care
- ↵
- ↵
- Detterbeck F
- ↵
- Hubbard RB,
- Baldwin DR
- ↵ Department of Health. Equity and excellence: liberating the NHS 2010London: DH
- ↵ British Thoracic Society. The burden of lung disease 20062nd ednLondon: BTS
- ↵
Article Tools
Citation Manager Formats
Jump to section
Related Articles
- No related articles found.
Cited By...
- A realist synthesis of multicentre comparative audit implementation: exploring what works and in which healthcare contexts
- European Respiratory Society guideline on various aspects of quality in lung cancer care
- Development of an Australia and New Zealand Lung Cancer Clinical Quality Registry: a protocol paper
- Can the completeness of radiological cancer staging reports be improved using proforma reporting? A prospective multicentre non-blinded interventional study across 21 centres in the UK
- Long-term survival in paraneoplastic Lambert-Eaton myasthenic syndrome