Does quality improvement improve quality?

The problems with quality improvement

Healthcare has increasingly been encouraged to use quality improvement (QI) techniques to tackle these operational defects (clearly, healthcare faces many other challenges but they may require different approaches). Capacity to improve quality is clearly critical to healthcare organisations; every organisation needs to be able to detect its operational (and other) problems and solve them using structured methods. For many problems (although far from all), that may mean using methods adapted from other industries, such as Lean and Six Sigma, or approaches developed within healthcare, such as the Institute for Healthcare Improvement's Model for Improvement. This widely used model combines measurement – using statistical process control, for example – with small tests of change (plan-do-study-act (PDSA) cycles).⁸ But despite the widespread advocacy for QI, the evidence that it produces positive impacts in healthcare has been very mixed, with many of the better-designed studies producing disappointing results.^9–14 A 2016 review concluded that Lean interventions, for example, do not have a significant association with patient satisfaction or health outcomes, but do have a negative association with financial costs and worker satisfaction, and inconsistent effects on process outcomes.¹⁵

What explains these discouraging findings is now the focus of growing interest. One explanation appears to lie in poor fidelity in the use of QI methods. For example, a 2014 review found poor reporting and adherence to the basic tenets of PDSA cycles in QI reports.¹⁶ More generally, what may happen is that the superficial outer appearance of the intervention or QI method is reproduced, but not the internal mechanisms (or set of mechanisms) that produced the outcomes in the first instance.^17,18 These effects may arise because what is implemented in practice may be diluted, distorted or diminished versions of the intervention, as has been found, for example, in relation to leadership walkrounds.^19,20

Secondly, much QI work continues to be undertaken in the form of time-limited small-scale projects, perhaps conducted as part of professional accreditation requirements. Some of the achievements of this work are striking, but caution is needed. One risk is that QI becomes an activity largely assigned to professionals in training, who rarely have the skills, resources or power to affect the kinds of changes that may be required. For instance, a problem with crowding in oncology outpatients may have its origins in a complex tangle of poorly designed or functioning processes (eg ensuring blood results are available on time), but diagnosing the cause and redesigning the workflow accordingly might need a dedicated team with specialist training in ergonomics and the clout to support the changes needed; these are not resources usually available to junior doctors or small QI teams. They may therefore come up with a small fix or workaround that fails to solve the true problems and, in so doing, may introduce new risks.

Another risk is that of encouraging ‘projectness’²¹ – a sense that QI is a series of bounded, time-limited events rather than a continuous commitment, and overly focused on ‘innovation’ rather than replication. Treating QI as a series of local projects may increase the tendency for wheel reinvention – different ‘solutions’ to the same problem. Undoubtedly, this expansion of overlapping efforts in part reflects the relative novelty of QI in healthcare. But it requires urgent attention, not least because ill-coordinated improvement may, ironically, intensify the problem of locally-specific work processes, routines and tasks that only apply in their context of origin. Multiple ill-coordinated small-scale QI projects may, accordingly, degrade rather than improve the ability to achieve improvements across healthcare as a whole.²² Moreover, as attention shifts from one project to another, the gains achieved in the first project may attenuate, a phenomenon that has been termed the ‘improvement evaporation effect’.²³

A third, and linked, problem is the ongoing failure to cumulate and share learning from QI efforts. The NHS continually loses learning, and this is an urgent problem. Although proper evaluation is essential to advancing the science of improvement,²⁴ those who introduce local QI interventions are sometimes so convinced that the change introduced is positive that they may eschew evaluation.²⁵ When people do come up with good ideas and test them rigorously, the learning may be difficult to share and challenging for others to discover – in part because the learning is never reported or, if it is reported, it is not in an accessible form. When people come up with ideas that don't work, the learning is even more likely to remain obscured. These problems contribute significantly to wheel reinvention and to waste of time and energy. Yet traditional medical research funding mechanisms and publishing norms are poorly aligned with the imperative to evaluate, curate and make available experiences (positive and negative) and outcomes of both QI methods and QI interventions. Even when QI is reported, it tends to be poorly described.²⁶ It therefore remains difficult to even find out about a success or a failure elsewhere, let alone to know what was really done and with what outcomes.

A further challenge lies in the ongoing emphasis on specific interventions as the keys to QI, perhaps particularly when those interventions are valorised as magic bullets.²⁷ The dynamic interplay between intervention and context means that it is often difficult, and indeed not always helpful, to separate intervention from context²⁸ to the extent that transplanting a programme in its entirety from one setting to another is rarely straightforward.²⁹ Excessive attention to QI interventions in the narrow sense – eg huddles, bundles, checklists and other popular tools – risks overlooking the impact of context on intervention implementation and, perhaps more importantly, the critical role of context itself as generative of safety and quality. Very often, the kind of place that has come up with the idea for doing huddles and has been able to implement and sustain them is also the kind of place that has all of the other characteristics that facilitate quality and safety. The notion that the huddle – or anything else – is then a plug-and-play ‘solution’ is consequently misguided – the features of context (clarity of vision, infrastructure, organisational systems, values, skills and so on) that made it work in the first place need to be reproduced too. Healthcare organisations differ markedly from factory production lines, just as human bodies are not ‘widgets’. Acknowledging and attending to the social and cultural context is vital if improvement interventions are to work.

The tendency to attribute effects to interventions (rather than interventions and contexts working together) is further exacerbated by the problem that the forces that create positive conditions for quality and safety may be invisible to those who create them or may not be possible (or straightforward) to articulate. This makes it difficult for others to reproduce or recreate them. The intervention as described in published reports may offer only a partial account of the reasons why the success was achieved. Foregrounding a specific intervention, no matter how well characterised, as the explanation for the outcomes may risk rendering invisible the important mechanisms that contribute to the achievement of those outcomes. The result is a theoretically deficient approach to improvement that may rely on ‘magical thinking’.³⁰

Many of these challenges can be illustrated by looking at the example of sepsis management. For patients with suspected sepsis, organisations are encouraged to do a ‘bundle’ of six clinical activities within 1 hour:

1. deliver high-flow oxygen

2. take blood cultures

3. administer empiric intravenous antibiotics

4. measure serum lactate and send full blood count

5. start intravenous fluid resuscitation

6. commence accurate urine output measurement.

Delivering on each one of these goals requires a supporting infrastructure, ranging from role clarity through to sufficient well-maintained equipment. For example, obtaining a serum lactate with a rapid turnaround time requires optimised equipment and organisational systems, as well as staff with the right expertise available at the right time. Making all of these things happen requires high-level skills in operations design but may also require all kinds of other skills in implementation, including negotiating for clarity about roles and responsibilities, managing professional or managerial resistance to reconfigurations of tasks, delivering high-quality training and so on.

It is probably not necessary for each individual organisation to invest the effort in figuring out all of the tasks and activities needed to achieve each of the goals. Nor is it likely that all organisations will have all of the necessary expertise to come up with good solutions. However, if a good solution is found, it may help others because it can be shared and give them a head-start. Such a solution will need to go beyond the narrow specifics of a well-bounded, easily describable intervention and encompass the range of facilitating conditions – infrastructural, technological, social, and maybe even cultural – that have often been relegated to the category of ‘context’, but which are themselves vital to the success of efforts to improve. It is also important that the solutions reached are broadly similar across organisations, so that once a practitioner has learned the system once he or she will know broadly what to do next time. It may be disastrous, for example, if the system for alerting professionals of the availability of a test result varies from one setting to another because they may rely on being alerted in a particular way, with the potential for delay if it does not happen.

Overcoming the challenges

Conflicts of interest

The authors have no conflicts of interests.