ABSTRACT
The European Society of Cardiology (ESC) and UK National Institute for Health and Care Excellence (NICE) have recently published guidelines for investigating patients with suspected coronary artery disease (CAD). Both provide a risk score (RS) to assess the pre-test probability for CAD to guide clinicians to undertake the most effective investigation. The aim of the study was to establish whether there is a difference between the two RS models. We retrospectively reviewed records of 479 patients who presented to a UK district general hospital with chest pain between August 2011 and April 2013. The RS was calculated using ESC and NICE guidelines and compared. From the 479 patients, 277 (58%) were male and the mean age was 60 years. The mean RS was greater using NICE guidelines compared with ESC (66.3 vs 47.9%, 18.4% difference; p<0.0001). The difference in mean RS was smaller in patients with typical chest pain (13.0%). When we divided the cohort based on NICE criteria into ‘high’- and ‘low’-risk groups, the difference in the mean RS was 24.3% in the ‘high’-risk group (p<0.001) compared with 2.8% in the ‘low’-risk group. The UK NICE risk score model overestimates risk compared with the ESC model.
Introduction
Patients suspected of having stable coronary artery disease (CAD) presenting with new onset chest pain can be investigated by numerous diagnostic modalities. National and international guidelines are published at intervals to help guide clinicians to undertake the most effective investigations. The National Institute of Health and Care Excellence (NICE) and European Society of Cardiology (ESC) have recently published their guidelines.1,2 Both guidelines provide a risk score (RS) to assess the pre-test probability for CAD derived from the Duke clinical score and the Diamond–Forrester method, respectively.3,4 The ESC RS is based on age, gender and typicality of pain, while NICE in addition differentiates patients into ‘low’- and ‘high’-risk groups depending upon pre-existing risk factors for CAD (diabetes mellitus, smoking history and hyperlipidaemia).
NICE recommends cardiac computed tomography (CT) for patients with low RS. They recommend an initial calcium score and only if the calcium score is above zero to proceed to CT coronary angiogram (CTCA). They recommend functional imaging tests, such as dobutamine stress echocardiography (DSE) or myocardial perfusion scintigraphy (MPS), for patients with intermediate RS and invasive coronary angiography (ICA) for patients with a high RS.1 The ICA is the most expensive of these tests at £1,241 compared with CTCA £164, DSE £288 and MPS £373.5 Our previous studies demonstrated that the NICE RS overestimates incidence of disease.6 Hence we have locally expanded the use of CTCA to patients with intermediate RS and the functional imaging tests to patients with high RS. Our aim in this study was to establish whether there is a difference between the ESC and NICE RS models.
Methods
Patients
We retrospectively analysed data between August 2011 and April 2013 of all patients seen in rapid access chest pain clinics at a UK district general hospital, that sees 525 patients per year. Consecutive patients with suspected stable CAD were included after excluding patients with either non-cardiac chest pain or previously established CAD (Table 1).
Baseline characteristics of population and the population size in each sub-group.
Information including patient demographics, baseline cardiac history, CAD risk factors and symptoms were obtained. Typical chest pain was defined as: (1) constricting discomfort in the front of the chest, in the neck, shoulders, jaw, or arms; (2) precipitated by physical exertion; and (3) relieved by rest or GTN within about 5 minutes.1 Atypical chest pain was classed as two factors defining typical chest pain. Non-anginal chest pain was classed as one (or zero) factors defining typical chest pain. Both the ESC and NICE guidelines RS were calculated on all patients and compared.
We attempted to establish factors contributing to, if not accounting for any variation between the NICE and ESC RS by comparing males vs females, typical vs atypical chest pain, NICE ‘high’-risk vs ‘low’-risk category, number of CAD risk factors, investigation-specific negative results vs positive results (CTCA, DSE, MPS and ICA), and all negative results vs positive results, as outlined in Tables 2 and 3.
Mean NICE and ESC risk scores (PTP) for all subgroups.
Mean NICE and ESC risk scores for all non-invasive investigations and invasive coronary angiogram.
Cardiac CT
Patients were beta-blocked by the referring clinician (atenolol 50 mg) and/or intravenously with metoprolol (5–30 mg) prior to cardiac CT aiming to achieve a heart rate of <60 bpm. All CTCA was performed on a 64-slice LightSpeed VCT XTe GE scanner (GE Healthcare) with prospective gating using the commercially available protocol (SnapShot Pulse, GE Healthcare). Significant CAD on CTCA was defined as >50% diameter stenosis.
Dobutamine stress echocardiogram
All patients were scanned using a Philips IE33 echocardiography machine. The images were acquired by a senior sonographer and reported by a consultant cardiologist. Intravenous dobutamine was administered via a syringe pump starting with 10 μg/kg/min and increased in increments up to a maximum of 40 μg/kg/min. Boluses of atropine up to a maximum of 1 mg were added if the target heart rate of 85% of maximum predicted was not achieved. Sonovue contrast was used at the discretion of the cardiologist. A new regional wall motion abnormality in two adjacent segments, of which only one can be an apical segment, was defined as a positive test.
Myocardial perfusion scintigraphy
Studies were performed under standard departmental protocols with a conventional gamma camera with sodium iodide detector and single photon emission computed tomography with technetium radiotracer. Images were acquired at rest and after stress, which was induced with dipyridamole (0.56 mg/kg over 4 minutes), followed by injection of the radiotracer. All images were reviewed jointly by a radiologist and a cardiologist. A new perfusion defect in two adjacent segments was defined as a positive test.
Statistical analysis
Statistical analyses were performed using SPSS version 18.0 (SPSS, Inc). Variables were tested for normality using the Kolmorgorov–Smirnov test. Values are expressed as either mean ± standard deviation or percentages, as appropriate. Differences between group means were compared using independent t-tests or Mann–Whitney U tests (for normally and non-normally distributed variables respectively). The χ2 test or Fisher's exact test was used as appropriate to test group differences of proportions. Bland and Altman plots were utilised to determine bias between the two pre-test probability risk scores.
Results
A total of 479 consecutive patients met the inclusion criteria and were included in the study. 277 patients (58%) were male and 202 (42%) were female. The mean age of the cohort was 59.7±11.7 years (Table 1).
For the whole study population the mean RS using the NICE model was 66.3% and for the ESC model 47.9% (difference 18.4%; p<0.0001). Bland-Altman plot demonstrates the mean bias of 18.4% (Fig 1).
Bland–Altman graph comparing NICE PTP vs ESC PTP for all patients. Mean bias 18.4% (95% CI 16.9–19.8). CI = confidence interval; ESC = European Society of Cardiology; NICE = National Institute for Health and Care Excellence; PTP = pre-test probability.
When we reanalysed the data based on gender, the divergence in risk scores remained with a difference in females of 19.8% (p<0.0001) and in males of 17.4% (p<0.0001). When we reanalysed the data based on typicality of chest pain the systematic difference between the two risk scores remained, but was lower for typical chest pain (13.0%; p<0.0001) (Table 2).
When patients were divided according to NICE classification of ‘low’- and ‘high’-risk groups based on the presence of cardiovascular risk factors, the difference was 2.8% (p =0.44) in the ‘low’-risk group and 24.3% (p<0.0001) in the ‘high’-risk group. This demonstrates that the classification into ‘low’- and ‘high’-risk groups is the main difference between the two risk-score models.
We further assessed whether reserving the classification of ‘high’ risk for patients with two or three CAD risk factors would make a difference to the RS classification (Fig 2). The divergence between the RS remained. However, there was a slight convergence in difference of mean RS in patients with three CAD risk factors (18.0%; p =0.02).
Mean NICE and ESC risk scores according to number of cardiovascular risk factors outlined by NICE guidelines. Statistical p values are representative of comparison between mean NICE and ESC risk scores. ESC = European Society of Cardiology; NICE = National Institute for Health and Care Excellence.
The data were further analysed based on the imaging modality selected for investigation. The same large systematic difference between the two RS remained but was relatively smaller for patients who underwent CTCA (13.9%; p<0.0001). When we analysed the data based on the outcome of the non-invasive investigations, the discrepancy between the mean RS remained. Importantly, the RS for all patients with negative investigation remained high in both NICE and ESC RS models, at 63.7% and 45.4% respectively (Table 3).
Ninety patients had positive non-invasive investigations, of these 63 were referred for ICA (± functional flow reserve). In these patients that underwent ICA, the difference in RS was 19.5% (p<0.0001). This is demonstrated by a Bland–Altman plot (Fig 3). Of these 63 patients, 40 had significant CAD on ICA. There was a convergence of the difference in those who had positive ICA with a difference between RS of 16.6% (p<0.0001).
Bland–Altman graph comparing NICE PTP vs ESC PTP for all patients who underwent invasive coronary angiography. Mean bias 19.5% (95%CI 16.1–22.8). CI = confidence interval; ESC = European Society of Cardiology; NICE = National Institute for Health and Care Excellence; PTP = pre-test probability.
Discussion
Our results show that there is a significant difference between the two RS models used in the UK, NICE and ESC guidelines. The RS was significantly higher when calculated using the UK NICE model across all subgroups, except in the NICE ‘low’-risk group category. Both guidelines overestimate the prevalence of CAD, highlighted by the high RS in patients who had negative non-invasive investigations (Table 3). However, the overestimation of the prevalence of CAD was considerably smaller in the ESC RS model. The systemic overestimation in both the Diamond–Forrester method, used by ESC, and Duke clinical score, used by NICE, is likely to be due to the fact that the risk scores were developed on tertiary centre patients in the USA, from data that is 30–40 years old that does not reflect current disease prevalence in primary care populations.
The cause for the difference in NICE and ESC risk scores is multi-factorial and includes disparity between the Diamond–Forrester method and Duke clinical score.3,4 Both of these were established in symptomatic patients referred for ICA. The Duke clinical score incorporated CAD risk factors and used a ≥75% angiographic stenosis severity as a cut off, whereas the Diamond–Forrester method did not incorporate CAD risk factors and used a ≥50% stenosis as a cut off. Recent studies have shown the Diamond–Forrester method to overestimate prevalence of CAD. Hoilund-Carlsen et al, in a cohort of 187 patients, showed that over half of patients referred for ICA with typical chest pain, who had RS >80%, based on the Diamond–Forrester method, had unobstructed coronary arteries.7 A more recent study by Wasfy et al, involving 114 patients, who underwent CTCA, demonstrated a significant improvement in risk prediction when using the Duke clinical score compared with the Diamond and Forrester method.8 The CONFRIM registry conclusively demonstrated that the Diamond–Forrester method overestimates the prevalence of CAD by up to three-fold compared with the observed prevalence, in 14,048 patients who underwent CTCA.9
Recently Genders et al published an updated version of the Diamond–Forrester method, in a multicentre cohort from 14 hospitals, based on patients with chest pain who were referred for ICA. They demonstrated that by revising the predictive value of age, gender and typicality of chest pain in the Diamond–Forrester method, they could partially rectify the systematic overestimation of the RS model.10 The ESC guidelines made an attempt to use contemporary data by using this modified and improved version of the Diamond–Forrester model. This is likely to explain why the ESC RS performed better than the NICE RS. However, the ESC RS still overestimates risk as it is still based on a high-risk population already selected for ICA.
Limitations
We collected consecutive patient data retrospectively. Our study population was derived from a single UK district general hospital. We did not strictly adhere to the investigation modalities outlined by UK NICE guidelines, CTCA was used in patients with intermediate RS and functional imaging tests were used in patients with high RS. This was due to our previous studies which demonstrated that the NICE RS overestimated the risk of CAD.6 However, this did not influence or bias our results as we compared the risk scores from two different guidelines and not their choice of investigation.
Conclusion
Our study demonstrates that the UK NICE RS model, based on the Duke clinical score, overestimates risk by an 18.4% compared with the ESC RS model, based on a modified Diamond–Forrester method. This is highly relevant as the NICE guidelines recommend that patients with a PTP score >61% should be investigated with ICA. ICA is costly and is associated with a 0.1% risk of mortality. If NICE adopted the ESC risk score, it would result in fewer patients having to undergo ICA as a first-line test. Refining the NICE RS in future guidelines, will be critical as the RS determines the choice of investigation.
- © Royal College of Physicians 2015. All rights reserved.
Article Tools
