Article Text
PDF
Abstract
Objective To evaluate the effects of Quality and Outcomes Framework (QOF) incentivised case finding for depression on diagnosis and treatment in targeted and non-targeted long-term conditions.
Design Interrupted time series analysis.
Setting General practices in Leeds, UK.
Participants 65 (58%) of 112 general practices shared data on 37 229 patients with diabetes and coronary heart disease targeted by case finding incentives, and 101 008 patients with four other long-term conditions not targeted (hypertension, epilepsy, chronic obstructive pulmonary disease and asthma).
Intervention Incentivised case finding for depression using two standard screening questions.
Main outcome measures Clinical codes indicating new depression-related diagnoses and new prescriptions of antidepressants. We extracted routinely recorded data from February 2002 through April 2012. The number of new diagnoses and prescriptions for those on registers was modelled with a binomial regression, which provided the strength of associations between time periods and their rates.
Results New diagnoses of depression increased from 21 to 94/100 000 per month in targeted patients between the periods 2002–2004 and 2007–2011 (OR 2.09; 1.92 to 2.27). The rate increased from 27 to 77/100 000 per month in non-targeted patients (OR 1.53; 1.46 to 1.62). The slopes in prescribing for both groups flattened to zero immediately after QOF was introduced but before incentivised case finding (p<0.01 for both). Antidepressant prescribing in targeted patients returned to the pre-QOF secular upward trend (Wald test for equivalence of slope, z=0.73, p=0.47); the slope was less steep for non-targeted patients (z=−4.14, p<0.01).
Conclusions Incentivised case finding increased new depression-related diagnoses. The establishment of QOF disrupted rising trends in new prescriptions of antidepressants, which resumed following the introduction of incentivised case finding. Prescribing trends are of concern given that they may include people with mild-to-moderate depression unlikely to respond to such treatment.
- PRIMARY CARE
- MENTAL HEALTH
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
-
Rigorous quasi-experimental design, demonstrating policy effects on patient populations within a sample of general practices, that appears broadly representative on key parameters.
-
Further insights gained from comparison of trends in patient populations targeted and non-targeted by intervention.
-
Relatively high ‘signal-to-noise’ ratio inherent in use of routinely recorded data may have diminished the magnitude of observed effects.
-
The absence of a control population of practices, making it hard to rule out possibility that concurrent national and local initiatives contributed to observed trends.
-
Lack of data on patient outcomes, such as recovery from depression or the appropriateness of treatment.
Background
Long-term physical conditions are associated with a high prevalence of depression; people with diabetes or coronary heart disease (CHD) have a twofold to threefold increased lifetime risk.1 ,2 Such comorbidity can make depression hard to recognise,3 ,4 worsens the prognosis of both conditions1 ,5 ,6 and increases healthcare and societal costs.1 ,7
The UK National Institute for Health and Care Excellence (NICE) recommends case finding for depression in people with long-term physical conditions.8 ,9 The Quality and Outcomes Framework (QOF) for general practice was established in 2004 and correspondingly rewarded case finding for depression in all patients with a diagnosis of CHD or diabetes over 2006–2013 (QOF years 3–9). This indicator was known as ‘QOF DEP1’ and defined as, “the percentage of patients on the diabetes register and/or the CHD register for whom case finding for depression has been undertaken on one occasion during the previous 15 months using two standard screening questions.”10 A designated clinical code indicating the use of these questions was recorded in the patient record whenever the Patient Health Questionnaire-2 was administered, irrespective of the responses. Practices were reimbursed according to the proportion of patients with a record of case finding in the preceding 15 months. Payment thresholds were set at achievements of 40–90% of eligible patients until 2012, and 50–90% during 2012–2013. The indicator had a value of eight points from 2006 to 2010 and six points from 2010 to 2013. Each point was worth £133.76 in 2012–2013, the final year of incentivisation. This incentivised case finding has now been withdrawn from the QOF because of doubts over benefits.11
The impact of this policy has been uncertain. The effectiveness of financial incentives in changing clinical behaviour is limited12 and pay-for-performance schemes often have unintended adverse consequences.13 More specifically, a systematic review concluded that advances in quality of care for long-term conditions included in UK QOF were modest.14 There are few rigorous evaluations of the effects of pay-for-performance given that controlled comparisons are rarely acceptable to policymakers. Two interrupted time series evaluations of QOF have not shown any sustained effects on processes of care or clinical outcomes.15 ,16 While there are no coded data prior to the introduction of the case finding indicator, at face value the QOF did incentivise a change in practice given that around 86% of patients with diabetes and CHD have been coded as screened at least every 15 months since its inception.17 Yet there is no evidence that case finding for depression, whether in the presence18 or absence of coordinated care systems,19 ,20 improves patient outcomes. A cohort study found a greater likelihood of a new diagnosis of depression and initiation of antidepressant treatment in the 28 days following QOF-incentivised case finding21; the longer-term effects on the whole population eligible for case finding are unknown. There may be further unintended effects on populations with other long-term conditions not targeted by incentivised case finding. Examining quality of care across a number of conditions, Doran et al22 found that improvements associated with QOF incentives occurred at the expense of small detrimental effects on aspects of non-incentivised care.
We evaluated the effects of incentivised case finding on new depression-related diagnoses and new prescriptions of antidepressants in patient populations with long-term conditions targeted or not by financial incentives.
Methods
Study design
We used an interrupted time series design to evaluate the effects of incentivised case finding while accounting for underlying secular trends. We also compared trends in depression diagnosis and treatment between those patient populations targeted by incentivised case finding (diabetes and CHD) and other patient populations with long-term physical conditions not targeted by incentivised case finding (hypertension, epilepsy, chronic obstructive pulmonary disease (COPD) and asthma). Our rationale was that we would not expect outcomes in the non-targeted group to diverge from underlying secular trends.
Practices and participants
We invited all 112 general practices in Leeds to share anonymised patient data via the Information in General Practice Team of the then National Health Service primary care trust. No distinction was made between users of different electronic records systems. Compared with English indicators the physical health of people in Leeds is generally worse and levels of deprivation are higher.23 Recorded depression in adults is similar (both around 11%)24 as is performance on the QOF incentivised case finding indicator in our final year of data collection (87% for Leeds over 2011–2012 compared with England average of 86%).17 ,25 We sought data on patients with diabetes and CHD targeted by case finding and data from other patients with the four comparator and non-target, long-term physical conditions from QOF registers. Patients with conditions in both targeted and non-targeted groups were excluded from non-targeted group analysis to avoid double counting. Therefore, any change in outcomes in the non-targeted group could not be attributable to individuals being screened because they had a targeted condition.
Data collection
We collected retrospective, electronic data from February 2002 through April 2012 for patients aged 18 years and over. Data were extracted through Morbidity Information Query and Export Syntax (MIQUEST) software used for collecting data from general practice clinical computing systems in a consistent and comparable way. The tool utilises a query language that incorporates security and confidentiality safeguards; pseudoanonymisation supports the extraction of patient level information but ensures it is not attributable to individual patients.26 Participating practices consented to the extraction of anonymised patient data and did not need to take any further action.
We recognised that the diagnosis of depression was likely to be under-recorded in clinical records because of factors such as diagnostic uncertainty and patient preference. The recording of certain diagnostic Read Codes, such as ‘depressive disorder’, automatically triggers alerts for further assessments required by QOF. Failure to meet these targets reduces practice income and hence coding behaviour may have changed. We therefore also searched for use of more sensitive but less specific Read codes, such as ‘low mood’ or ‘depressed mood’, that are not assessed by the QOF and included these in our main outcome of diagnosis. We excluded codes related to postnatal depression.
Data on the prescription of licensed antidepressant drugs listed in British National Formulary section 4.3 were collected, with the exception of antidepressants judged by clinicians involved in the project (RF, AH, SLA, KM) to be more commonly prescribed for other indications (eg, amitriptyline and nortriptyline for neuropathic pain).27
A complete list of clinical codes for each outcome measure is available as an electronic web appendix.
Data analysis
The denominators comprised the numbers of patients on practice registers for each financial year (starting 1 April) targeted by incentivised case finding (diabetes and CHD) and those not targeted (hypertension, epilepsy, COPD and asthma). We assumed that registered long-term condition populations would be relatively stable over each year. We took the number of registered long-term condition populations per practice as constant over each QOF year. This permitted a more parsimonious model to facilitate interpretation.
For each targeted and non-targeted patient group, we analysed trends in new depression-related diagnoses and antidepressant prescribing. We also examined the uptake of case finding for depression. We recognised that these trends could relate to changes in coding as well as clinical practice; we mainly used their outputs to guide interpretation of the main outcomes. Data were aggregated by month for each of the 65 practices so that each time series is 123 months long (February 2002 to April 2012). Analysis was carried out at the practice level using a binomial regression based on the calculated numerators and the available denominators. Discontinuities were modelled at key dates: April 2004 for the introduction of QOF and April 2006 for the introduction of incentives for case finding for depression. A further discontinuity was introduced at April 2007 to isolate exceptional behaviour noted during the QOF year April 2006 through March 2007. Our focus and interest was on the long-term sustained effect seen after the introduction of case finding incentives rather than the immediate change. To avoid bias from this first year (2006/2007), rates were permitted to be different in that year, thereby isolating it from the sustained effect we sought to assess. For each time period (February 2002 to March 2004; April 2004 to March 2006; April 2006 to March 2007; April 2007 to April 2012) the model has an overall constant and slope. Specific slope terms were dropped when they were found not to be statistically significant from zero at the 5% level.
Fitting seasonal effects improved the model but added complexity. As reference and intervention periods were integer multiples of complete years, there would be no perturbation of level or slope if explicit seasonality terms were not included, but rather seasonality was encompassed within the error term. As the profile of seasonality appeared to change from the reference period to the intervention period and vary in the group with targeted interventions compared with the group for other long-term conditions, this option was selected to yield the clearest effect in the model. The model can be expressed as
Let 
and 
be random variables representing the number of diagnoses at practice 
in month 
for targeted and non-targeted patients, respectively. Then
1where 
is the relevant denominator for practice 
in month 
, and 
is the corresponding rate of diagnosis. Using a logit link function in the generalised regression, we model the rate 
with
2and
3where 
is an indicator variable for the year 2006/2007 and 
is an indicator for the intervention period, that is, after the year 2006/2007. Note that a random intercept 
is included to account for clustering within practices. Slope terms were also added where appropriate. The open source software R 2.12.0 64 bit version was used for all statistical analysis.28
Results
We recruited 65 (58%) of 112 Leeds practices. Their 2012 QOF registers indicated that they served 37 229 patients with diabetes and CHD targeted for case finding for depression and 101 008 patients with other long-term conditions not targeted. Table 1 provides data on all English practices and compares characteristics of recruited and not-recruited practices.
Characteristics of general practices in England and those in Leeds that did and did not share data for the study based on data published in 2012
Overall, the practices recruited were larger; however, we found no significant differences in Indices of Multiple Deprivation, or total QOF scores. The majority of practices used one clinical computing system by the end of data collection. Tables 2 and 3 summarise the annual incidences of case finding, depression-related diagnoses and prescription of antidepressants by count and rates per 100 000 patients, for targeted and non-targeted patients.
Annual numbers of case finding, new depression-related diagnoses and new prescriptions of antidepressants in Leeds over 2001–2012 for conditions targeted or not by incentivised case finding
Annual incidences of case finding, new depression-related diagnoses and new prescriptions of antidepressants (per 100 000 patients) in Leeds over 2001–2012, for conditions targeted or not by incentivised case finding
Practice-level analysis found significant increases in new coded case finding following the initiation of incentives, also reflected in aggregated city-wide level trends (figure 1). The exceptional rise in 2006 reflects first coding in patients with existing diagnoses of diabetes and CHD. Comparing the period April 2004 to March 2006 with April 2007 to March 2012, rates of case finding increased in the targeted population from 0.07 to 7.45/1000 per month (OR 99.76; 95% CI 83.15 to 119.68) and in the non-targeted population increased from 0.1 to 0.78/1000 per month (OR 7.54; 6.91 to 8.24).
Rates of coded case finding for depression in patients with conditions targeted or not by incentivised case finding, 2002–2012.
Binomial regression of the practice level data confirmed statistically significant rate increases in new depression-related diagnoses in both patient populations. In targeted patients, the diagnosis rate increased from 21 to 94/100 000 per month between the periods 2002–2004 and 2007–2012 (OR 2.09; 1.92 to 2.27). In non-targeted patients, the rate increased from 27 to 77/100 000 per month (OR 1.53; 1.46 to 1.62). In neither of these periods was the slope statistically significant from zero, that is, the rates can be assumed to be constant during these periods. Figure 2 shows these trends aggregated at a city level with fitted constants and slopes, indicated by dashed lines. Figure 3 shows the city-level trends for new antidepressant prescribing with fitted constants and slopes. Rates of prescribing increased over the full period of observation. During the period after QOF was introduced but before incentives (April 2004 to March 2006), the slopes for both populations flattened to zero (p<0.01 for both groups). For targeted patients, the slopes before the introduction of QOF and after the exceptional year were similar (Wald test for equivalence of slope, z=0.73, p=0.47). For non-targeted patients, the slope for the latter period was less steep (Wald test for slope, z=−4.14, p<0.01). All Wald tests for slopes were undertaken using practice level data.
Rates of new depression-related coded diagnoses in patients with conditions targeted or not by incentivised case finding, 2002–2012.
Rates of new antidepressant prescribing in patients with conditions targeted or not by incentivised case finding, 2002–2012.
Discussion
Incentivised case finding increased rates of new depression-related diagnoses in patients with CHD and diabetes and, to a lesser extent, in those with non-targeted long-term conditions. The establishment of QOF disrupted rising trends in new prescriptions of antidepressants; these resumed following the introduction of incentivised case finding, although there was a modest deceleration in antidepressant prescribing for non-targeted conditions. Rates of new prescriptions for antidepressants exceeded those for depression-related diagnoses.
Quasi-experimental evaluations of QOF have found no sustained effects for other clinical indicators.14–16 Financial incentives in primary care tend to have modest effects on relatively simple clinical behaviours such as risk factor recording or test ordering.12 The nature of targeted clinical behaviours is likely to influence the effectiveness of incentives.29 ,30 Given that the QOF incentives directly rewarded case finding, we sought and found evidence of changed clinical practice ‘downstream’ to case finding. Previous research has found associations between case finding for depression and new diagnoses and antidepressant prescribing.21 ,31 However, our analysis of longitudinal data demonstrates policy effects at a population level and highlights the importance of accounting for secular trends and additional insights from comparative data.
The mechanisms by which rates of depression-related diagnoses increased remain unclear. The spike in diagnoses immediately following incentivisation probably reflects coding patterns before general practitioners began to realise they would trigger alerts for further assessments required by QOF when recording depression-related diagnoses. Similar phenomena have been observed in the first years of new QOF indicators.32 Following the introduction of incentivised case finding, rates of new depression-related diagnoses rose in non-targeted long-term conditions, coincident with only a modest rise in recorded case finding in these patients. Incentivised case finding may have directly affected pathways of care or, more generally, increased awareness of the higher risk of depression in all patients with long-term conditions. A combination of these explanations seems likely for two reasons. First, we found strong evidence of seasonality for coded case-finding but not for new diagnoses or prescribing. Second, our parallel ethnographic study of general practices demonstrated the absence of a systematic approach to following up and managing screen-positive cases.33 It remains uncertain how the QOF and other payment for performance systems work.34
The interpretation of prescribing trends is more challenging. Taking pre-QOF trends into account, new prescriptions of antidepressants in patients with long-term conditions plateaued following the introduction of QOF before resuming the underlying trend in targeted conditions when incentivised case finding for depression was introduced. This plateau effect seems compatible with a view that the initial introduction of QOF diverted attention from psychosocial aspects of long-term condition care towards achieving biomedical targets.35 It is also consistent with a longitudinal analysis of QOF in English general practice, which found lower overall achievement rates for non-incentivised indicators compared with predicted values than for incentivised indicators.22 Arguably, this might not represent a detrimental unintended consequence in the case of a potentially over-medicalised condition such as depression.36
The causes of on-going secular increases in antidepressant prescribing have been debated.37 ,38 Hypotheses include poor compliance with clinical guidelines that do not recommend prescribing in the more commonly encountered mild-to-moderate depression,39–41 an increase in duration of antidepressant prescribing in line with clinical guidelines rather than an increase in the number of patients prescribed for,42 and the intensifying effect of QOF on prescribing patterns.43 Our data included only the first prescription of any antidepressant for each patient, indicating that our observed trends are attributable to greater numbers of patients being treated rather than extended periods of prescribing. Therefore, our analysis supports the explanation that incentivised case finding perpetuated the rise in antidepressant prescribing because of a perceived need for clinical action over and above referral for counselling or watchful waiting.
The rate of antidepressant prescribing in this study exceeded the rate of diagnosis of depression in targeted and non-targeted groups; this trend was also reported by Burton and colleagues.21 The limited use of clinical codes in the diagnosis of depression is recognised. Rather than a lack of diagnostic accuracy, it probably reflects how clinical coding is not always a part of routine practice and how general practitioners pragmatically prescribe according to symptoms and responses to treatment rather than diagnostic categories.44 ,45
While we drew on published guidance in conducting this interrupted time series,46 ,47 we identified seven main limitations. First, the high ‘signal-to-noise’ ratio inherent in the use of routinely recorded data may have diminished the magnitude of observed effects.48 Second, the true denominator for the binomial regression varies monthly as patients exit the denominator population after undergoing incentivised case finding. There are also variations due to patients dying and leaving the practice. We used annual QOF reports for the denominator values and took them to be constant for that year. As the denominator is large compared with the number screened, the error of the model will be small. Third, we were unable to examine patient outcomes, such as recovery from depression, nor the appropriateness of treatment. We explored the use of routinely collected referral data but these were unreliably recorded and prone to temporal changes in coding practices. Fourth, targeted patients with diagnoses of diabetes and CHD may include individuals with a greater number of comorbidities than non-targeted patients.49 Depression is more prevalent in patients with a greater number of physical comorbidities,50 ,51 suggesting we were more likely to identify depression-related diagnoses in this group. Fifth, our analysis is based on one geographical area with a response rate of 58%. However, the characteristics of practices participating in the study were broadly similar to those for England and the non-participating practices. Sixth, observed trends may also have been related to changes in practice computerised record systems. Leeds practices began migrating to The Phoenix Partnership (TPP) SystmOne after 2006 until it became the majority provider in 2012 (table 1). The choice of clinical computing system is associated with variations in practice QOF performance.52 Seventh, given the absence of a control population of practices, it is possible that concurrent national and local initiatives may have contributed to our observed trends. NICE issued a clinical guideline on depression in 2004, which was subsequently revised in 200941; even allowing for delayed diffusion or anticipatory effects, it is unlikely to explain any changes we observed from 2006 onwards. Nor do the introduction of the Improving Access to Psychological Therapies programme in Leeds from 2008 to 2009 onwards or publication of the NICE clinical guideline on depression in adults with a chronic physical health problem in 2009 offer plausible alternative explanations.53 ,54 Furthermore, the isolation of the exceptional year when case finding incentives were first introduced permits us to infer with confidence that we observed sustained higher rates of diagnosis.
Given the sustained promotion of case finding for depression across a range of long-term conditions and for carers,8 ,9 ,55 there is a need for clearer guidance to optimise the pathway and outcomes of care for case finding-detected depression, including limiting antidepressant prescribing to patients most likely to benefit. Any effects of incentivised case finding need to be considered alongside costs. On the basis of payments offered under the 2012–2013 UK QOF contract and without considering opportunity costs, we estimate that case finding for depression in CHD and diabetes cost over £6 million per annum56 in the context of the £1 billion total estimated cost of QOF each year. These costs, the limited benefits we found, and the withdrawal of incentivised case finding for depression demonstrate the risk of rolling out policies in the absence of rigorous supporting evidence. Although policymakers express frustration when debates about evidence appear to hold back service improvement,57 there are hazards in following assumptions about how and whether apparently simple but deceptively complex interventions such as incentivised case finding work.58
The impact of the withdrawal of QOF incentivised case finding for depression is not yet known. A retrospective longitudinal study suggested levels of performance remain stable across a range of clinical activities following the removal of QOF incentives, although all indicators studied were indirectly or partly linked to activities that remained incentivised.59 The longer-term effects of completely withdrawing an incentive, such as case finding for depression, on clinical behaviour is unknown and merits further research.
Acknowledgments
The authors thank Dr Paul Lord, University of Leeds, for compiling practice average and England average demographic characteristics.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Footnotes
-
Contributors RF and AH conceived the project. RF was principal investigator. KM and SLA designed the study. KM and AMR were responsible for running the project. RW was responsible for statistical analyses. All authors interpreted the data and findings. KM wrote the first draft of the manuscript. RF commented on the first draft and all authors commented on further revisions. KM is guarantor of the paper.
-
Funding This paper summarises independent research funded by the National Institute for Health Research (NIHR) under its Research for Patient Benefit (RfPB) Programme (Grant Reference Number PB-PG-0110-21046).
-
Competing interests All authors report grants from National Institute for Health Research under its Research for Patient Benefit Programme, during the conduct of the study.
-
Ethics approval This study was approved by the East Midlands—Derby 2 Research Ethics Committee (reference 11/EM/0144).
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
Data sharing statement Full dataset and statistical code available from the corresponding author at k.l.mclintock@leeds.ac.uk. Consent was not obtained but the presented data are anonymised and risk of identification is low.
Linked Articles
- General practice / Family practice