ABSTRACT
Background The Ottawa subarachnoid haemorrhage (SAH) rule and the Emerald SAH rule are clinical decision tools to aid in the decision for computed tomography (CT) of the head in patients attending an emergency department (ED) with acute non-traumatic headache. The objective of this study was to analyse the performance of these rules in a contemporary UK cohort.
Methods We performed a retrospective external validation study. Patients undergoing CT of the head for the evaluation and treatment of non-traumatic headaches over a 6-month period in the ED at two tertiary centres were assessed. Each patient’s Ottawa rule and Emerald rule were calculated and compared with their final diagnosis.
Results The cohort consisted of 366 patients and there were 16 cases of SAH (based on CT findings or the presence of xanthochromia in cerebrospinal fluid). The Ottawa rule identified 288 patients requiring CT of the head. The sensitivity of the Ottawa rule was 100% (95% confidence interval (CI) 71–100%) and the specificity was 22% (95% CI 18–27%). The Emerald rule identified 267 patients who required CT, and achieved a sensitivity of 81% (95% CI 54–96%) and a specificity of 27% (95% CI 23–32%).
Conclusions The Ottawa SAH rule correctly identified all patients with SAH in this contemporary cohort. The Emerald rule did not perform as well in this cohort and is unsuitable for clinical use. The Ottawa rule is a useful tool to aid in the decision for CT of the head in patients presenting with acute non-traumatic headache to the ED.
Introduction
Spontaneous subarachnoid haemorrhage (SAH) represents an important differential diagnosis to consider in patients presenting with sudden onset headache, as it is associated with significant morbidity and mortality. Non-traumatic subarachnoid haemorrhage (SAH) affects 6–12 people per 100,000 every year.1,2 Large numbers of patients present to the emergency department (ED) with headache, approximately 1–2% of all ED presentations.3 Sudden onset headache accounted for 0.8% of emergency presentations in one large UK cohort study and, in a European cohort, thunderclap headache accounted for approximately 0.15% of all ED presentations.4,5
The need for a safe and efficient method to identify patients at a high risk of SAH and to identify patients that would not benefit from a CT of the head is of paramount importance. Clinical decision tools may provide a means with which to achieve this, of which, two are currently described in the literature: the Ottawa SAH rule and the Emerald SAH rule.
The Ottawa SAH rule was developed by Perry et al on the basis of data derived from a multicentre cohort study in Canada.6 It takes into account clinical features that were deemed a high risk for SAH, including age ≥40 years, neck pain/stiffness, witnessed loss of consciousness, onset during exertion, thunderclap headache and limited neck flexion on examination. If any one or more of these clinical features are present, the patient must undergo computed tomography (CT). A recent review of the literature yielded four studies across four countries externally validating the rule, involving a total of 3,317 patients.7 Sensitivity across all studies was 100% and specificity ranged from 7.6–13.6%.8–11
The Emerald SAH rule was derived by Kimura et al from a multicentre cohort study in Japan.9 It uses objective, quantitative measures that aim to remove inter-observer variability, namely systolic and diastolic blood pressure, blood sugar and serum potassium. Based on a cohort of 1,317 patients, the authors reported 100% sensitivity and 14.5% specificity.
These tools have the potential to provide a cost-effective method to identify high-risk patients for SAH and to optimise use of resources by avoiding unnecessary CT. It is therefore important to evaluate their validity and usefulness. This two-centre cohort study aimed to examine the sensitivity and specificity of the Ottawa rule and Emerald rule and to provide external validation of these tools in a UK cohort.
Methods
Study design
All consecutive CT of the head examinations taking place in two UK EDs (King’s College Hospital, London, and Addenbrooke’s Hospital, Cambridge) were analysed over a 6-month period from January to June 2016. A retrospective review of all CT of the head requests was undertaken. Inclusion criteria were any patient aged 16 years or older and undergoing CT of the head for the investigation of SAH. Requests that specifically included SAH, sudden onset headache, thunderclap headache or ‘worst headache of life’ were included. Patients with a history of trauma, with a headache onset >14 days from presentation or those with a second or subsequent CT of the head within the study period were excluded. Requests which included subdural, hypertensive or intracranial haemorrhage as the working diagnosis were also excluded. Two authors divided and reviewed all CT of the head requests over the study period using these inclusion and exclusion criteria. Marginal cases were discussed between the authors and included or excluded by consensus. Patients’ electronic notes were then retrospectively reviewed to gather clinical information regarding the headache history, examination findings, biochemical results and to record the final diagnosis in each case. The diagnosis of SAH in this cohort was identified either by the presence of SAH on non-contrast CT of the head as per the consultant radiology report, or the presence of xanthochromia on lumbar puncture (LP) performed at least 12 hours after the onset of headache. This was defined by a value greater than 0.007 absorbance units (AU) for net bilirubin absorbance or a value greater than 0.02 AU for oxyhaemoglobin.
Analysis
Each patient’s clinical information was then used to calculate their Ottawa rule and Emerald rule. Patients with any one of the criteria in the scoring systems were deemed to have a positive score and, therefore, require a CT of the head. The criteria for the Ottawa rule are age ≥40 years, neck pain/stiffness, witnessed loss of consciousness, onset during exertion, thunderclap headache and limited neck flexion on examination. The criteria for the Emerald rule are systolic blood pressure, diastolic blood pressure, blood sugar and serum potassium. These parameters were taken from the patient’s initial presentation to ED and are the first recorded values. Each patient’s Ottawa and Emerald rule was compared with the final diagnosis in order to calculate the performance of each tool in the total cohort. Sensitivity, specificity, negative predictive value, positive predictive value and the corresponding 95% confidence intervals (CIs) were calculated for each clinical decision tool in this cohort. Subgroup analysis was also performed in the cohort of patients in whom headache reached maximal intensity within 1 hour, as per the original inclusion criteria for the Ottawa rule.6 The rate of CT of the head in each cohort analysing the Ottawa rule was recorded and compared with the rate of CT of the head that would be recommended by the Ottawa rule. The relative percentage change in CT of the head rate was then calculated by dividing the change in CT of the head rates by the original CT of the head rate. Data analysis was carried out in R version 3.1.1 and the gdata and epiR packages.12,13 This study received ethical approval from the research and audit committees at each institution.
Results
The total cohort consisted of 366 patients, the clinical characteristics of which are displayed in Table 1. There were 212 women, accounting for 58% of the cohort. The median age was 45 years, with 57% of the cohort over the age of 40 years. SAH was diagnosed in 16 patients (4.3%). The diagnosis was made on CT in 12 patients and on LP in four patients. The median time from headache onset to presentation was 1 day, ie half the cohort presented with 1 day of headache onset. The majority of patients were diagnosed with a benign headache (28.7%), ie a headache which is not due to an underlying structural cause. This included those patients with a headache yet to be classified and referred to neurology for further assessment. Other diagnoses included migraine (24.3%), tension headache (6.2%), hypertensive headache (2.5%), intraparenchymal haemorrhage (1.4%), musculoskeletal pain (1.1%), subdural haematoma (0.8%) and primary brain neoplasm (0.3%).
Patient cohort, n=366
In the total cohort, the Ottawa rule was positive in 288 patients and the Emerald rule was positive in 267, see Table 2. The performance of these clinical decision tools in the total cohort and in the subgroup analysis are illustrated in Table 3. The Ottawa rule achieved a sensitivity of 100% (95% CI 71–100%) in the total cohort with a specificity of 22% (95% CI 18–27%). The Emerald rule, however, achieved a lower sensitivity of 81% (95% CI 54–96%) with a corresponding specificity of 27% (95% CI 23–32%). This corresponds to a negative predictive value of the Ottawa rule of 100% (95% CI 93–100%) and a positive predictive value of 6% (95% CI 3–9%). The negative and positive predictive values for the Emerald rule were 97% (95% CI 91–99%) and 5% (95% CI 3–8%), respectively. The number of patients who had either a positive Ottawa rule or Emerald rule was 335.
The components of the Ottawa and Emerald subarachnoid haemorrhage rules and their distribution within the cohort
Performance of the Ottawa subarachnoid haemorrhage rule and Emerald subarachnoid haemorrhage rule
The use of these two tools combined achieved a worse performance than the use of the Ottawa rule alone. A two-step strategy utilising the Ottawa rule followed by the Emerald rule achieved a sensitivity of 81% (95% CI 54–96%), specificity of 41% (95% CI 36–46%), negative predictive value of 98% (95% CI 94–100%) and positive predictive value of 6% (95% CI 3–10%).
The results of these risk calculators within the subgroup of patients whose headache reached maximum intensity within 1 hour (n=65) were also analysed. The Ottawa rule again achieved 100% sensitivity (95% CI 28–100%) and a specificity of 20% (95% CI 11–32%). The Emerald rule demonstrated a sensitivity of 100% (95% CI 28–100%) and a specificity of 28% (95% CI 17–100%). This corresponds to a negative predictive value of the Ottawa rule of 100% (95% CI 64–100%) and a positive predictive value of 8% (95% CI 2–18%). The negative and positive predictive values for the Emerald rule were 100% (95% CI 73–99%) and 8% (95% CI 2–20%), respectively.
Discussion
The Ottawa rule demonstrated a 100% sensitivity in this external validation cohort and a specificity of 22%. In other words, the use of this tool missed no cases of SAH and would have allowed the clinician to accurately exclude the diagnosis of SAH in 22% of patients, and safely avoid a CT of the head. The 22% specificity demonstrated in this external validation cohort exceeds that of the original internal validation cohort, which was 15%.6 The rate of SAH in the present study was 4.3%, which is lower than that of the Ottawa rule cohort and the Emerald rule cohort, which had rates of SAH of 6.1% and 17.7%, respectively. The implication of this lower SAH rate in the present study is that, de facto, there is a greater specificity of the Ottawa rule in our cohort.
The Emerald rule demonstrated a worse performance in this cohort in comparison with the Ottawa rule. In the creation of the Emerald rule, the authors did not, a priori, define the variables to be included but only included those that were significant on univariate testing and then considered relevant. This suggests that the Emerald rule would over-fit their data and hence why this external validation demonstrates suboptimal performance. The Emerald model achieved a sensitivity of less than 100% in the present study, and missed the diagnosis of SAH in three patients within the total cohort. Two of these cases were diagnosed on CT of the head, while one case was diagnosed following lumbar puncture. Another explanation for the suboptimal performance of the Emerald rule is the use of objective criteria only, in an attempt to remove the subjectivity associated with each patient’s clinical history. However, the subjective features are necessary in order to most accurately diagnose SAH, as evidenced by the superior performance of the Ottawa rule. Kimura et al stated that the use of a two-step strategy of the Ottawa rule followed by the Emerald rule may allow for a greater performance.9 However, in this cohort, the use of the Ottawa rule followed by the Emerald rule would still have missed 3 cases of SAH; in our cohort, we cannot confirm this is an appropriate strategy for risk stratification of patients with suspected SAH. The first goal of a risk calculator used in the investigation of SAH is to achieve a 100% sensitivity and to ensure no cases of SAH are missed, as this can lead to disastrous consequences for the patient. In this respect, the Ottawa rule achieves this goal and the Emerald rule does not.
In the subgroup analysis of patients presenting with headache reaching peak intensity within 1 hour, the Ottawa rule and the Emerald rule both achieved a 100% sensitivity and did not miss any cases of SAH. This result is especially meaningful in the context of the high accuracy of the CT of the head for a diagnosis of SAH if performed within 6 hours of headache onset.14 However, these data must be interpreted with caution. Only 65 patients in the cohort had a headache of maximal severity within 1 hour, and so the performance on the Emerald rule in this smaller group may be misleading. The Ottawa rule however, maintained its 100% sensitivity in the total cohort and so the Ottawa tool is more robust for use in clinical practice. Using the less stringent inclusion criteria in the total cohort, the Ottawa rule still did not miss a case of SAH. The Ottawa rule has demonstrated in this study that its use in patients may not be restricted to those presenting with an acute onset headache maximal in intensity within 1 hour. In patients with a peak headache intensity within 1 hour, the incidence of SAH is estimated at 6%.6 While headache is the presenting symptom in 70% of SAH patients, and there is a sudden onset in approximately 50% of patients.15,16 In the present study, we have analysed a contemporary cohort of patients that were referred for a CT of the head with the clinical question querying SAH. Therefore, the scope of the Ottawa rule may be wider than previously described.
The Ottawa rule has undergone internal and external validation in a number of cohorts, see Table 4. External validation has taken place in over 4,000 patients and, with the exception of one study, has demonstrated a sensitivity of 100%.8–11,17–19 Specificity has been lower and has ranged from 7.6% to 44%. These external validation studies, together with the results from our present study, represent a robust external validation of the Ottawa rule and provide an evidence base for its introduction into clinical practice to aid in the decision for CT of the head in cases of suspected SAH. Indeed, our study supports the conclusions drawn by the American College of Emergency Physicians’ (ACEP’s) clinical policy for acute headache, particularly regarding its high sensitivity but limited specificity.12 Furthermore, our study represents the first external validation of the Ottawa rule in a UK cohort. Conversely, the performance of the Emerald rule had yet to be externally validated and the present study is the first to examine its performance in a new patient cohort.
Performance of the Ottawa subarachnoid haemorrhage rule on internal and external validation
The only study to demonstrate a less than 100% sensitivity of the Ottawa rule on external validation was undertaken by Cheung et al in a Chinese population.18 There were three missed cases of SAH in the study population of 500. Unfortunately, Cheung et al did not examine the performance of the Emerald rule in their study cohort. Because the Emerald rule was produced in an Asian cohort, it may potentially be of greater use in a similar population.
The use of the Ottawa rule is limited by its specificity. The potential impact of the Ottawa rule on neuroimaging rates is analysed in Fig 1. In total, there were eight studies that included rates of CT of the head in the original patient population and the potential rate of CT of the head if the Ottawa rule was applied. This included the external validation studies discussed earlier and two other studies from Matloob et al and Yiangou et al.20,21 These authors analysed an earlier version of the Ottawa rule and so the increase in CT rates in these studies is not exact but indicative only. There was an increase in the need for CT of the head following the application of the Ottawa rule in five of the eight studies, ranging from <1% to 100%. The other three studies demonstrated decreased rates of neuroimaging ranging from 2% to 27%. However, in a prospective study from Perry et al, there was no increase in rate of neuroimaging following the introduction of the Ottawa rule to clinical practice.19 Taken together, although the Ottawa rule is limited by its specificity, it does not necessarily lead to increased rates of neuroimaging. Indeed, given the consistent demonstration of the rule’s sensitivity across several studies, its role in reducing neuroimaging and subsequent investigation rates for low-risk patients may be more apparent in larger cohorts. In summary, the use of the Ottawa rule in clinical practice has the potential to accurately stratify patients for CT of the head and subsequent investigations if CT of the head is negative (namely lumbar puncture and angiography), while not missing any cases of SAH. Nevertheless, the combination of the Ottawa rule with other strategies to increase specificity while maintaining sensitivity would greatly improve the rule’s usability, and has been highlighted by ACEP as an area of future research.12
The potential impact of the Ottawa rule on neuroimaging rates.
Although beyond the scope of this article, there arises another important clinical question of whether to perform lumbar puncture in patients with a positive Ottawa rule, following a negative CT of the head. This is a separate question, however, because of the low specificity of the Ottawa rule, it is one that will be frequently encountered clinically. The creators of the Ottawa rule are currently developing a second tool, which aims to calculate each patients risk of SAH following a negative CT of the head.22 It is important to note that in the present study, there were 327 patients with a negative CT for SAH and no alternative diagnosis and only 104 of these patients subsequently underwent LP for cerebrospinal fluid analysis, which equates to approximately 32%. Therefore, although all of the 68% of patients who did not undergo LP were not given a diagnosis of SAH, it is unclear how many of these patients were truly normal or whether the cohort contains some false negatives secondary to inadequate investigation.
Due to the retrospective design of this study, clinical characteristics that were not recorded in the patients’ clinical notes were assumed to be absent, rather than specifically asked for and recorded as present or absent. Only Perry et al have utilised the Ottawa SAH rule in a prospective fashion.10,19 All other external validation studies are merely applying the rule retrospectively. The present study is also limited in that patients with a headache peaking in intensity within 1 hour may not have been recorded accurately and so this subgroup may be incomplete. Furthermore, patients with a milder or dull headache may have difficultly reporting the time to peak intensity. Here, we have examined those patients who have undergone CT of the head for a suspected subarachnoid haemorrhage. We have not analysed the clinical information of patients presenting to the ED with acute headache who have not undergone CT of the head. This means that the potential increase or decrease in CT of the head rates with the use of the Ottawa rule in this patient cohort cannot be calculated.
Conclusion
The Ottawa rule corrected identified all patients with SAH in this cohort and is a highly sensitive clinical decision tool. The Emerald rule did not perform as well in this cohort and is unsuitable for clinical use. In patients presenting to the ED with an acute non-traumatic headache, the Ottawa rule can be used as an aid in the decision for CT of the head in clinical practice, however, this tool is limited by its specificity.
Summary
- © Royal College of Physicians 2021. All rights reserved.
