ABSTRACT
Neuroradiologists provide quality-assured neuroimaging reports. We developed the use of a neuroimaging team meeting to provide second-opinion reporting by neuroradiologists on neuroimaging that had previously been reported by general radiologists. Neuroimaging from selected patients was reviewed at the meeting. Where there were discrepancies between an original report from a general radiologist and the report obtained from the meeting involving a neuroradiologist, we classified the discrepancies, recorded the scan modality involved and used the data to assess temporal trends in discrepancy rates. Over 4 years, 562 patients (312 women, 250 men, mean age 50.6 [SD 17.3] years) were studied. Agreement occurred for 396 (70.5%) patients. Discrepancies that were not clinically important occurred for 60 (10.7%) patients. Clinically important discrepancies were found for 106 (18.9%) patients: missed lesions for 47 (8.3%) patients and misinterpretations for 59 (10.5%) patients. Cerebrovascular disease was the most common reason for a recommendation of neuroimaging review at a meeting. Scan modality did not influence the frequency of discrepancies. Discrepancy rates decreased with time (chi-squared test for linear trend p=0.015), while the frequency of neuroradiologists’ recommendations for new investigations was stable at one in seven patients. Neuroimaging team meetings can facilitate improvements in neurology diagnoses.
Introduction
Delayed, missed and incorrect diagnoses occur for many reasons: system-related errors, mistakes in interpreting diagnostic tests and cognitive errors in clinical medicine, including neurology.1,2 The interpretation of diagnostic tests (pathology, radiology and clinical laboratory) has an established error rate of 2–5%.2 Diagnostic accuracy is becoming more important in radiology because advances in technology are facilitating the potential for increasingly accurate diagnoses with less intervention.3,4 The 2015 publication from the Institute of Medicine on Improving Diagnosis in Health Care recognises the scale and implications of inaccurate or delayed diagnoses.5 The report provides guidance on improvement strategies, including an emphasis on teamwork.5
Description of the problem
Because of the vast number of recognised neurological diagnoses, neurology (perhaps more than other medical disciplines) has seen increasing diagnostic advances that are facilitated by timely and expert imaging reports. In frontline services, neurologists provide more diagnoses than non-neurologists6 (sometimes shortening lengths of stay7), claiming over 95% diagnostic accuracy8 when aided by timely and quality-assured neuroimaging. Such standards have been reported mostly from teaching hospitals.6,7 In reality, however, most neurological patients are cared for at the secondary care level, in district general hospitals. Provision of service equity in a quality-assured healthcare system mandates accurate diagnoses for all,2 but there has been little research on service standards in this area. Concerns that service standards may be determined by geography9 have, however, prompted the need for quality-improvement interventions. Harnessing appropriate skills to maximise the value and equity of healthcare provision for a population is a goal of modern medicine and a responsibility of clinicians.10
Available knowledge
It is generally accepted that neuroimaging is best reported by neuroradiologists.11 Discrepancies in reporting from academic neuroradiologists are rare.12 There is evidence that written reports from neuroradiologists can provide important revisions to neuroimaging initially reported by general radiologists,13–16 albeit at the risk of increasing time to diagnosis. Accessing neuroradiologists for quality-assured reporting of selected neuroimaging can enhance local neurology services in a general hospital.16 Commissioned healthcare services often neglect to fund routine practice with evidence-based advances in multidisciplinary standards.
Rationale
Neuroradiologists are usually based in tertiary care hospitals and, historically at least in the UK and Ireland, have had little formal input into secondary care neurology services. Nevertheless, their formal written opinions have contributed to quality improvements.14,16 The Hawthorne effect17 may, however, have contributed to a bias in at least some of these studies. (The Hawthorne effect describes how an individual modifies behaviour as a response to being observed. For example, a neuroradiologist's written report may differ if he or she is aware that the report is the subject of a study.) Real practice assessment of a service-based team meeting is therefore required to measure any quality improvement that was achieved through regular meetings between neurologists and neuroradiologists for the management of selected neurological patients.
There has been little reported evidence of the role of team meetings employing teleneuroradiology to facilitate second-opinions from neuroradiologists for neurological patients in a general hospital.
Specific aims
For selected neurological patients attending a general hospital, we sought to determine the frequency of differences between an original opinion from a general radiologist and the report from a team meeting involving neurology and neuroradiology staff. We also sought to measure whether these meetings were associated with a temporal trend in the rate of discrepancy findings. Finally, we assessed the discrepancy data to determine the proportion of patients in different diagnostic categories and to assess different scan modalities.
Methods
Context
Following the establishment and development of a neurology service in our hospital in 2003, the need for quality-assured neuroimaging reporting emerged.13,15,16 Encouraged by the Royal College of Radiologists, a weekly neuroimaging team meeting was set up.
Intervention
Neurologists selected neuroimaging scans when (1) there was doubt or ambiguity in the initial report, (2) a general radiologist reported a recommendation for a neuroradiologist's second-opinion; or (3) neurologists thought an alternative diagnosis or differential diagnosis existed. A patient archiving and communication system (PACS) permitted weekly meetings to take place at either the secondary or tertiary care hospitals to minimise any increased time to diagnosis.
Study of the intervention
At each meeting, a neuroradiologist's opinion was transcribed. A meeting report was generated with both sets of reports (original general radiologist's report and neuroradiologist's comments or revised report). Outcomes were recorded using a discrepancy scale devised by Zan et al11 in which score 1 was agreement without any discrepancy; score 2 was a discrepancy in detecting a clinically unimportant abnormality; score 3 was a discrepancy in interpreting a detected but clinically unimportant abnormality; score 4 was a failure to detect a clinically important abnormality; and score 5 was a clinically important discrepancy in interpreting a detected abnormality. Clinical importance was defined by a change in diagnosis that affected prognosis, treatment, follow-up or onward referral to other specialists. Two neurologists (MOM and FM) independently scored the discrepancies. Where disagreement in a discrepancy score occurred, a consensus emerged from discussion.
Measures
Patients’ demographic profiles were recorded. The primary outcome measure was the proportion of selected patients with clinically important discrepancies recorded between 1 January 2013 and 31 December 2016.
Secondary outcome measurements included clinically unimportant discrepancies and all discrepancies. Another secondary outcome was the number of patients for whom further neuroimaging study was recommended by the neuroradiologist at the meeting.
Discrepancies were categorised as previously described.11 The temporal trend in discrepancy rate was measured over the 4 years of the study.
Analysis
Mean age and standard deviation were calculated. The discrepancy scores were analysed for each year and for the whole study. The number and proportion of patients for each outcome score were calculated with 95% confidence intervals, a proportion obtained using the Newcombe procedure with a correction for continuity.18 Fisher's exact test was used to compare categorical values. The chi-square for trend test was used to assess the change in discrepancy rate over time.
Ethical considerations
The trust for the district general hospital approved the study as audit and quality improvement. As such, the study did not require formal ethics approval. All patients verbally consented to participation. This quality improvement program is reported using the revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0).19
Results
Initial steps of the intervention
Among a total of 562 patients, the study included 312 women and 250 men, mean age 50.6 (SD 17.3) years. For each year, between 124 and 162 patients had their neuroimaging reviewed. Outcome category 1, agreement without any discrepancy, was achieved for 396 patients or 70.5% of reports (Table 1). Consensus (between neurologists MOM and FM) in classifying discrepancies was reached for the remaining 166 patients. Clinically important discrepancies occurred in 106 patients or 18.9%. Figure 1 highlights some missed lesions, which were clinically important and examples of detected but misinterpreted lesions, which were also clinically important.
Frequency and classification of agreement and discrepancies in neuroimaging
Examples of clinically important missed lesions and misinterpreted lesions. (a) CT scan of brain showing left Virchow Robin space, which was misinterpreted as a lacunar infarct. (b) MRI of brain with subdural collections (b1, which was identified in the initial report) and sagging of the brainstem (b2, not identified) suggesting intracranial hypotension. (c) MRI of brain right hemisphere gyrus with signal change of unclear significance (c1) that enlarged over 4 years (c2), making low-grade tumour the probable diagnosis. The enlargement had not been recognised. (d) MRI brain showing decussation of the superior cerebellar peduncles, which was misinterpreted as a midbrain infarct. (e) Right frontal low signal on gradient echo MRI of brain consistent with contusion but misinterpreted as a cavernoma. Clinically important lesions missed on initial reports included (f) right cerebellar stroke, (g) cerebral venous sinus thrombosis showing absence of contrast at the confluence of the venous sinuses, (h) torcula meningioma and (i) a spinal cord lesion at T1–2 consistent with demyelination. CT = computed tomography; MRI = magnetic resonance imaging
Clinically unimportant discrepancies occurred in 60 patients or 10.7%. Examples of misinterpreted but clinically unimportant lesions included non-specific white matter change which was diagnosed in the meeting as inflammatory demyelination. Examples of missed and clinically unimportant discrepancies included developmental venous anomaly and pineal cysts.
There was no difference in the distribution of missed and misinterpreted lesions between CT-scan-only patients (14 of 58 patients) and MRI-scanned patients (152 of 504 patients, Fisher's exact test, p=0.584).
Vascular imaging abnormalities were the most frequent reason for selecting patients for a second opinion. These patients also represented the largest category for clinically important discrepancies (34 of 179 or 19%). Table 2 shows agreement and discrepancy frequencies stratified for aetiological classification.
Aetiology of agreement and discrepancy scan reports
Contextual elements of the outcomes that interacted with the intervention
The discrepancies were fed back to general radiologists at the district general hospital during a monthly meeting at the district general hospital and in a written report from a neuroradiologist to provide an addendum report.
Observed associations between outcome, interventions and contextual elements
Year on year from 2013 to 2016 the total discrepancy rate declined from 38.9% to 22.6% (chi-square for linear trend p=0.015, Fig 2a). For clinically important discrepancies (missed and misinterpreted lesions), there was a similar temporal trend showing improvement from 23.5% to 11.3% (chi-square test for linear trend p=0.021, Fig 2b).
Temporal changes over four years in: (a) all neuroimaging discrepancies; (b) clinically important neuroimaging discrepancies; and (c) the frequency of neuroimaging recommendations for further investigation with 95% confidence intervals.
Unintended consequences such as unexpected benefits, problems, failures, or costs associated with the intervention
New recommendations for neuroimaging were reported for 78 patients (13.9%).
New recommendations varied between 8.1% and 20.7% (Fig 2c). No temporal trend was identified (chi-square test for linear trend p=0.86).
Details about missing data
Data were collected prospectively. All patients discussed at the neuroimaging team meetings over the 4 year study period were included in the study.
Discussion
Summary
This study demonstrates that the presence of a neuroradiologist in a weekly team meeting where the imaging for selected neurological patients from a district general hospital is assessed can identify a range of neuroimaging discrepancies. In addition, regular feedback has been associated with a temporal trend towards improvement of the quality of neuroimaging reporting from general radiologists in the same hospital. Finally, this type of multidisciplinary meeting also appears to provide a constant rate of clinically useful recommendations for neuroradiological follow-up.
Interpretation
Oncology has highlighted many attributes of teamwork: processes of care and increasingly better outcomes.20 Teamwork has also been shown to enhance the management of complex neurological conditions such as Parkinson's disease.21 Neurologists are pivotal in providing diagnosis for patients who have neurological presentations; neurologists provide more neurological diagnoses than generalists and these diagnoses have a high degree of reliability.8 Similarly, neuroradiologists provide the most reliable quality-assured reporting on neuroimaging.11 As for other medical specialties, we have demonstrated that a team approach at the secondary-care level can improve diagnoses for neurological patients.22
Until recently, there has been less emphasis on diagnostic error,5 including neurological diagnostic error.1 At least 5% of US outpatient adults experience a diagnostic error each year. These are not clinically insignificant issues as 6–17% of hospital adverse events are attributed to diagnostic error.5 Among the strategies highlighted by the Institute of Medicine to improve diagnostic error, our neuroimaging meeting has embraced the strengthening of teamwork, health information technology (with the use of PACS), measuring and learning from real-world practice and the promotion of a culture of diagnostic safety.23
Education interventions can be evaluated using the so-called four Kirkpatrick levels of evidence from an education program.24 For postgraduate medical staff these include: (1) a doctor's reaction or satisfaction; (2) a doctor's demonstration of learning knowledge, skill or attitude; (3) change in clinical behaviour; and (4) improved patient outcome. Our study suggests that general radiologists demonstrated improvements in Kirkpatrick levels 2 and 3. A recent review of educational interventions in neurology has shown that very few published studies exist in which an intervention can change clinical behaviour.25
There is emerging anecdotal evidence of the value of neuroradiology team meetings to provide feedback and improve communication, as has been acknowledged for individual case reports.26 Our study extends this work and suggests that such meetings can improve diagnostic accuracy. We anticipate subsequent improvement in patient care, but our study has not confirmed Kirkpatrick level 4 (improved patient outcome) which is the ultimate goal of any educational intervention. Daily time constraints on all clinicians compete with the implementation of quality-improvement projects. There is, however, increasing scrutiny of the real value of a health service for organisational quality.27,28 In addition, diagnostic error is the most common cause of neurological litigation.29 These factors are already exerting influence on the need for quality improvements at all levels of healthcare. The Global Health Commission has emphasised the interdependence of health care service and education.30 The goals of such interdependence include: methods to 'harmonise education and health systems’ as well as a shift 'from standalone institutions to networks, alliances, and consortia’. Neuroimaging team meetings can encompass these goals while improving diagnostic accuracy, in line with guidance from the Institute of Medicine's Improving diagnosis in health care.5
Limitations
The study was performed in a single hospital and used mostly one neuroradiologist. However, previous contributions from multiple neuroradiologists (including the principal neuroradiologist in this study) showed similar discrepancy rates.15 The proportions of patients in the different diagnostic categories are different from those obtained in second-opinion studies from tertiary centres,11 with fewer neoplastic and more vascular patients in our secondary-level care study. This reflects the disease burden within a general hospital where vascular neurology is by far the most frequent neurological disease in the community.31 Our findings may also apply to similar neurology services within the NHS, but this requires confirmation.
This type of study risks selection bias, although previous studies have provided similar discrepancy rates that have been externally validated.16 The broad selection criteria used by the neurology and radiology team have not been validated but have evolved with clinical need. The current lengthy study reflects real practice. Transcribed verbal opinions from a neuroradiologist may not be as detailed as written reports, but our study demonstrates their clinical usefulness by employing the Zan et al classification.11
Finally, we recognise that general radiologists, who also work on other aspects of radiology, will inevitably incur a degree of spectrum bias in neuroradiology. Total agreement may not be possible and the progress identified to date may flat-line. Future research will be required to test this hypothesis.
Conclusions
This study has demonstrated that the strategic commissioning of a neuroradiology team meeting can provide useful education for general radiologists in a district general hospital, thereby improving service standards in diagnostic accuracy, and may improve patient safety.
Author contributions
Mark McCarron designed the study, analysed the data and wrote the first draft.
Carrie Wade collated the data and revised drafts of the paper. Peter Flynn provided second opinions, produced the imaging figures and revised drafts of the paper. Ferghal McVerry helped design the study, validated the discrepancies and revised drafts of the paper.
- © Royal College of Physicians 2018. All rights reserved.
References
Jump to section
Related Articles
Cited By...
- No citing articles found.