Evaluation of a nudge intervention providing simple feedback to clinicians of the consequence of radiation exposure on demand for computed tomography: a controlled study
ABSTRACT
Computed tomography (CT) is readily available in developed countries. As one of the side effects includes an increased risk of cancer, interventions that may encourage more judicious use of CT are important. Behavioural economics theory includes the use of nudges that aim to help more informed decisions to be made, although these have been rarely used in hospitals to date. We aimed to evaluate the impact of a simple educational message appended to the CT report on subsequent numbers of CT completed using a controlled interrupted time series design based in two teaching hospitals in the UK. The intervention was the addition of a non-directional educational message on the risk of ionising radiation to all CT reports. There was a statistically significant reduction in the number of CT requested in the intervention hospital compared to the control hospital (-4.6%, 95% confidence intervals -7.4 to -1.7, p=0.002) in the 12 months after the intervention was implemented. We conclude that a simple, non-directional nudge intervention has the capacity to modify clinician use of CT. This approach is cheap, and has potential in helping support doctors make informed decisions.
Introduction
Computed tomography (CT) has undoubtedly been one of the major factors contributing to improvements in medical care over recent decades. As there is widespread variation in usage of diagnostic tests,1 it is likely that some are unnecessary, which in the case of CT exposes the patient to avoidable ionising radiation. The estimates of the magnitude of this risk are uncertain; an estimate from 2007 suggested that use of CT in that year may account for 1.5 to 2% of new cancers in the USA,2 although this would be expected to decrease as newer generation CT machines deliver less ionising radiation exposure. Clinicians are not always aware of the long-term adverse health consequences of CT,3,4 suggesting that this is an area with potential for improvement. It was recently demonstrated that there is almost a three-fold increase in the risk of receiving a CT during the period of transition from paediatric to adult medical care,5 and one interpretation of these data is that paediatric clinicians may be better in using alternatives to CT than their adult counterparts.
Nudge theory is a category of behavioural economics that aims to use non-directional interventions to modify decision making.6 This generally aims to use opportunities to provide information or context that may allow better informed decision making, at a relatively low cost in terms of money or time. We hypothesised that providing referring clinicians with a simple statement about the consequences of radiation exposure may modify their subsequent decision making with regard to future use of CT. We tested this hypothesis in a busy teaching hospital in the UK using a prospective, controlled study design using the demand for CT as the primary outcome measure.
Methods
Study design
We used a controlled interrupted time series design.
Study population
The setting was Royal Derby Hospital, UK (RDH), with Nottingham University Hospitals, UK (NUH) as a control hospital being the nearest large hospital serving a similar population. The Royal Derby Hospital is a busy acute medical hospital that has 1,100 beds.7 The study was an evaluation of a health service modification and no ethical approval was required.
Intervention
The intervention comprised the addition of the message below to the bottom of all inpatient and outpatient paper and electronic CT reports. It was designed to highlight the type of patient who is most at risk after exposure to ionising radiation.Message from the executive medical director: ‘Did you know that a chest, abdo and pelvis CT scan in a 20 year old female population is associated with approximately a 1 in 300 risk of subsequent cancer? The equivalent risk is much lower in 90-year-old men (less than 1 in 3000). Is there an equally effective alternative investigation that does not involve ionising radiation? If so, have you discussed all of the alternatives with your patient?’ http://www.xrayrisk.com/index.php
Outcome measures
Data on the number of weekly CT between February 2016 and February 2018 were collected, covering a period 12 months before the intervention was implemented to 12 months afterwards in RDH and NUH. A sensitivity analysis omitted CT of the head as these involve a lower ionising radiation dose.
Statistical analysis
Segmented regression was used for the interrupted time series analysis, using a generalised linear model and assuming a negative binomial error distribution, that is, we fitted a model which was appropriate to count data with ‘overdispersion’ where there is greater variability in the data than that normally expected in count data. We used the package tscount in R statistical software.8 We initially fitted a full model for a controlled interrupted time series, with:
terms for the pre-intervention trend – to allow for any underlying baseline trend in CT rates
baseline difference between hospitals – to allow for a difference in mean CT rates between hospitals at baseline
difference in the pre-intervention trend between hospitals – to allow for any difference in the underlying trend between hospitals
change in level for NUH – to allow for any change in CT rates in the control hospital post intervention due to confounding factors
change in trend for NUH – to allow for any change in trend post intervention in the control hospital due to confounding factors
difference in the post-intervention change in level between RDH and NUH – our measure of the effect of the intervention on the mean level of CT rates in the intervention hospital compared to any change in the control hospital
difference in the post-intervention change in trend between RDH and NUH – our measure of the effect of the intervention on the trend in CT rates in the intervention hospital compared to the control hospital.
We therefore hypothesised an impact model for the intervention which might involve an immediate change in level or a gradual trend change in CT counts in RDH, which were tested by the last two parameters described above. A dummy variable was fitted to allow for the annual drop in CT numbers over Christmas.
We also fitted parsimonious (simplest) models including only parameters that were statistically significant, identified by backward elimination, removing those terms consecutively that were least significant until all parameters were significant at the 5% level and no parameters were significant at the 5% level when added to the model.9,10 Models were checked for autocorrelation (that is, for the fact that observations near in time may be correlated). An autoregressive process of order 4 was added to the final model to allow for each observation to be correlated with the previous four observations.
Results
The weekly numbers of CT in RDH (intervention institution) and NUH (control institution) are shown in Fig 1. Rates were slightly higher in NUH than RDH, but the underlying baseline trends were comparable.
The results of the resulting segmented regression analysis are shown in Table 1. In the full model, there was a significant difference in numbers of CT at baseline between hospitals, and a significant trend for increasing rates of CT at baseline, but no significant difference in the baseline trend between hospitals (p=0.8), and the level (p=0.055) and trend (p=0.3) for NUH did not change significantly post-intervention. There was also no significant difference in the change in trend post-intervention (p=0.3), in other words there was no significant effect of the intervention on the trend in CT rates.
In the final parsimonious model, there were 29% fewer CT carried out in RDH than NUH pre-intervention (p<0.001), and a significant increasing trend in the number of CT at 0.1% increase per week in both hospitals. However, there was a statistically significant reduction in level of CT in RDH compared to NUH post-intervention of 4.6% (p=0.002). The results excluding head scans were very similar.
Discussion
Statement of principal findings
These data demonstrate that providing simple, non-directional feedback on the long-term consequences of exposure to ionising radiation is associated with a 4.6% decrease in the number of CT completed compared to the control hospital over 1 year.
Strengths and weaknesses of the study
This study has a number of strengths and limitations. The use of a prospective study design with a control group allowed the change in the number of CT delivered to be compared against the background of a trend of increasing demand for healthcare annually. The intervention was added to the radiology reporting system and the text of the message was placed underneath the CT report, which all clinicians see when reviewing the scan result. The message on the CT report was designed collaboratively within a multidisciplinary team and piloted, but we are unable to conclude that it is optimal, and there is likely to be room for improvement in both the content and presentation of the message. As a consequence of designing and delivering ‘nudge’ informational messages in the context of health services research, there were a number of factors that required to be taken into consideration such as the opinions of important stakeholders, such as the radiologists, and the limitations of the institutional information technology systems. The data that the educational ionising radiation message was based on are necessarily population based, as it is impossible to provide precise individual level data on radiation risk. The key assumption in the controlled interrupted time series is that the change in the level and/or trend in the outcome variable is presumed to be the same for both the control group and, counterfactually, for the treatment group had it not received the intervention. These hospitals were reasonably similar at baseline with similar increasing trends in CT at baseline, no significant change in level or trend in the control group at the point of the intervention, and we are not aware of any other reason to expect differences in the change in levels or trends over time between the hospitals other than the intervention. We are unable to exclude the possibility of a second intervention being implemented at the same time as our educational one, but as most of the study team work in RDH, this is unlikely. We have not conducted any sex or age stratified secondary subgroup analyses, as the primary hypothesis was that there was be a change in demand for CT at the institutional level, and we wished to avoid the risk of multiple hypothesis testing.
As there were 52 paired data points before and after the intervention, we have refrained from secondary data sub-analyses, to avoid the problem of multiple hypothesis testing within a limited number of data points, which is also relatively underpowered compared to our primary analysis. These findings represent a positive proof-of-concept study, and demonstrates that the ionising radiation educational message intervention is one that can be implemented, within the context of the UK health service. The definitive study would be a larger, randomised controlled trial of more healthcare institutions to provide proof that this intervention is generalisable to other similar hospitals. This would be a large undertaking, that would require substantial resources to deliver.
Meaning of the study: possible explanations and implications for clinicians and policymakers
Nudge theory has potential in helping doctors make more informed decisions when choosing the optimal imaging approach for their patients; weighing up the immediate benefits of diagnostic knowledge against the longer-term consequences. As the information provided is non-directional, and simply informative, the doctors’ clinical autonomy is not infringed upon, as it is added to the context of many years’ medical training and experience for many UK healthcare teams. Providing the information after a scan has been completed and reported informed future practice, and had been demonstrated to be effective in modifying physicians’ behaviour with regard to use of blood tests after cost feedback was provided with the results.11 This approach was preferred to the alternative of delivering the information at the time of deciding to do the scan, when it could impact on the efficient delivery on clinical care. In the UK, the decision to request a CT for a patient is generally made in a face-to-face discussion with the patient, when the benefits and costs can be weighed up, and a collective decision made between the physician and the patient. The CT examination is then subsequently requested, when the decision to scan has already been made. As a consequence, feeding back an educational message on the longer-term health effects of exposure to ionising radiation was the only viable intervention in this context, although point-of-care interventions can be implemented for other healthcare interventions, and have been effective in increasing prescribing of cheaper antibiotics when the price was presented at the point of antibiotics selection.12
Unanswered questions and future research
In summary, we have demonstrated that adding a simple educational ionising radiation awareness message to CT reports was associated with a 4.6% decrease in subsequent demand for CT compared to the control group or approximately 45 fewer CT delivered per week or 2,340 fewer CT over a year. This nudge approach has potential to improve patient care when used prudently. However, it is unclear in which settings it will be efficacious and more controlled intervention studies are required across a broad range medical areas to develop the evidence base. We anticipate that informing physicians of the long-term risks of ionising radiation is unlikely to negatively impact on healthcare delivery, but future studies should consider both the benefits and the risks of such interventions.
Funding
Funding for this study was provided by The Health Foundation.
- © Royal College of Physicians 2019. All rights reserved.
References
- ↵
- ↵
- ↵
- ↵
- ↵
- Thurley P
- ↵
- Thaler R
- ↵
- Derby Teaching Hospitals NHS Foundation Trust
- ↵
- Liboschik T
- ↵
- ↵
- ↵
- Fogarty A
- ↵
- Langley T
Article Tools
Citation Manager Formats
Jump to section
Related Articles
- No related articles found.
Cited By...
- Systematic review of clinician-directed nudges in healthcare contexts
- Development and evaluation of a brief educational cartoon on trainee clinicians awareness of risks of ionising-radiation exposure: a feasibility pre-post intervention study of a novel educational tool to promote patient safety
- Does cost feedback modify demand for common blood tests in secondary care? A prospective controlled intervention study
- Reducing demand for computed tomography
- Nudge nudge, wink wink