Effect of a “Lean” intervention to improve safety processes and outcomes on a surgical emergency unit
BMJ 2010; 341 doi: https://doi.org/10.1136/bmj.c5469 (Published 02 November 2010) Cite this as: BMJ 2010;341:c5469
This article has a correction. Please see:
- Peter McCulloch, reader in surgery1,
- Simon Kreckler, clinical research fellow2,
- Steve New, lecturer in operations management2,
- Yezen Sheena, core surgical trainee3,
- Ashok Handa, tutor in surgery4,
- Ken Catchpole, senior research fellow1
- 1Quality, Reliability, Safety and Teamwork Unit (QRSTU), Nuffield Department of Surgery, University of Oxford, Oxford, UK
- 2Said Business School, University of Oxford
- 3John Radcliffe Hospital, Oxford
- 4Nuffield Department of Surgery, University of Oxford
- Correspondence to: P McCulloch, QRSTU, Nuffield Department of Surgery, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK peter.mcculloch{at}nds.ox.ac.uk
- Accepted 12 September 2010
Abstract
Problem Emergency surgical patients are at high risk for harm because of errors in care. Quality improvement methods that involve process redesign, such as “Lean,” appear to improve service reliability and efficiency in healthcare.
Design Interrupted time series.
Setting The emergency general surgery ward of a university hospital in the United Kingdom.
Key measures for improvement Seven safety relevant care processes.
Strategy for change A Lean intervention targeting five of the seven care processes relevant to patient safety.
Effects of change 969 patients were admitted during the four month study period before the introduction of the Lean intervention (May to August 2007), and 1114 were admitted during the four month period after completion of the intervention (May to August 2008). Compliance with the five process measures targeted for Lean intervention (but not the two that were not) improved significantly (relative improvement 28% to 149%; P<0.007). Excellent compliance continued at least 10 months after active intervention ceased. The proportion of patients requiring transfer to other wards fell from 27% to 20% (P<0.000025). Rates of adverse events and potential adverse events were unchanged, except for a significant reduction in new safety events after transfer to other wards (P<0.028). Most adverse events and potential adverse events were owing to delays in investigation and treatment caused by factors outside the ward being evaluated.
Lessons learnt Lean can substantially and simultaneously improve compliance with a bundle of safety related processes. Given the interconnected nature of hospital care, this strategy might not translate into improvements in safety outcomes unless a system-wide approach is adopted to remove barriers to change.
Surgical wards are an area of clear vulnerability in current healthcare systems because patients are much less closely monitored and staff to patient ratios are much lower than in intensive therapy areas. In addition, patients usually stay on surgical wards during the time period in which the risk of specific operative complications is highest. It is now widely accepted that high technology surgical care causes unintentional harm to around 10% of inpatients, and research effort is focused on finding solutions to this problem.1 2 3 4
Treatment related harm appears to occur mostly through sequences of small, seemingly innocuous events that combine unpredictably.5 6 7 Corrective efforts commonly involve changing isolated aspects of systems in response to accidents,8 or initiatives to improve the reliability of single processes of care.9 However, substantial improvement is likely only if the system of care is redesigned to eliminate small contributory errors across a wide range of care processes. Attempts at radical system redesign to improve quality and safety across an organisation, which are relatively common in other industries, have been rare in healthcare.
Arguably, the most successful example of system redesign in industrial settings has been the Toyota Production System, or “Lean.”10 11 12 13 Defining Lean is difficult: it is in essence the elimination of waste through continuous improvement (box 1). Identifying waste leads inevitably to the need to define customer value, and reducing waste requires elimination of error. This approach has found widespread application in industry. In medical settings, there is extensive evidence of its benefits in improving efficiency, reducing costs, and improving patient satisfaction.14 15 16 17
Box 1: The five principles of “Lean”
“The five S’s” (sort; set in order; shine; standardise; and sustain): the re-ordering of the working environment to clarify and simplify process and to reduce time wasted in finding supplies and equipment18
Process mapping: identification of problems to facilitate directed measurement and improvement19
Error visibility: development of audit methods to make problems instantly obvious20 21
Elimination of waste: minimising waste and wasted effort to make work easier22
“Plan, do, check, act” (PDCA): introducing cycles that allow rapid refinement of interventions,23 and defining problems and solutions by measurement rather than assumption
Assertions are regularly made, largely on the basis of anecdote and logical inference, that using approaches similar to the Toyota Production System strategy will have substantial benefits for patient safety,24 25 26 27 28 29 30 31 32 but there is little robust empirical support for these claims. The available evidence tends to focus on dealing with single safety relevant processes, such as increasing safety concern reporting,33 preventing pharmacies from temporarily running out of drugs (“stock outs”),34 or improving administration of antibiotic therapy.35 Few studies have directly measured the impact of Lean on patient safety.36
The problem
We have previously highlighted the high rates of process non-compliance associated with typical rates of patient harm during surgical care on the surgical emergency unit at the John Radcliffe Hospital, Oxford.37 We found a more than 10% rate of treatment related harm in this setting associated with deficiencies in compliance with recommended practice for important care processes.
The surgical emergency unit at the John Radcliffe Hospital is a 38 bed acute surgical ward in a teaching hospital in the United Kingdom that receives all general surgical emergency admissions to the hospital, as well as a small number of elective surgical patients (4%). Outlying vascular, orthopaedic, and medical patients make up about 13% of the ward population. The unit is typically fully occupied, with an average turnover of 10 patients a day and a staffing ratio of one trained member of staff to six patients. The mean length of stay is 3.2 days.
The unit is served by 10 consultant teams involved in the general surgery on-call service, who represent the oesophagogastric, colorectal, hepatobiliary, breast, and endocrine surgery teams. Patients are reviewed by a consultant team the morning after admission and a care plan confirmed.
We developed the hypothesis that process redesign using Lean with a focus on improving safety relevant care processes might significantly reduce the risk of care related harm to patients on surgical wards.
Key measures for improvement
Our study collected the following two main types of data: data on the performance of specific safety relevant care processes; and outcome data on patient safety incidents.
Evaluation of safety relevant care processes
Seven routine care processes were selected to represent aspects of care important to the safety of surgical patients. Processes were selected if: (a) they had a clear relation to patient safety; (b) they were reliant on ward staff to ensure implementation or compliance; and (c) they were not directly related to another selected process. The selection decision was made after a literature review, examination of local guidelines, and a consensus exercise using a panel of nine experienced surgical and nursing staff members.
The processes selected were: (1) direct verbal communication between medical and nursing teams on daily rounds; (2) administration of prophylaxis for deep vein thrombosis; (3) correct prescription of medications; (4) use of alcohol gel for hand hygiene; (5) infection monitoring of intravenous lines (venous site infection protocol scoring); (6) monitoring of patients’ vital signs and recording of their risk scores (track and trigger scoring); and (7) fluid balance recording.
Evaluation of patient safety incidents
Data on patient safety outcomes were collected from cohorts of consecutive general surgical admissions in the four months before introduction of the Lean intervention and the four months after completion of the intervention. We evaluated harm and potential harm by prospective daily direct observation, using a previously described method.38 Briefly, patients were tracked daily by a surgical research fellow (SK) from admission until discharge, transfer, or death. Each day, clinical staff were interviewed, case notes examined, and ward rounds attended to establish expectations for the patient’s course over the next 24 hours. Actual events were reviewed the following day, and discrepancies with predicted events were analysed.
We used the National Patient Safety Agency, United Kingdom, definition of a patient safety incident: “Any unintended or unexpected incident that could or did lead to harm for one or more patients receiving NHS funded healthcare.”39 If actual harm occurred, the incident was recorded as an adverse event; if not, then a potential adverse event was recorded. Adverse events and potential adverse events were categorised using criteria adapted from the pre-screening criteria of the Harvard Medical Practice Study.39 An example of an adverse event would be a patient with pancreatitis who developed acute renal failure after not receiving proper fluid balance management over a weekend because he was difficult to catheterise and to cannulate. An example of a potential adverse event would be a young woman who waited 48 hours for her appendicectomy because of pressure on emergency theatre time, but whose appendix did not perforate and who made a smooth recovery.
Process of gathering information
We studied safety relevant care processes and patient safety incidents before and after introduction of the Lean quality improvement intervention. The observation, intervention, and re-observation phases lasted four months (May to August 2007), eight months (September 2007 to April 2008), and four months (May to August 2008), respectively.
Collection of data on safety relevant care processes
Compliance with the protocol for each process was monitored by repeated audits of all eligible patients present on the ward on a single day.37 For some processes (for example, prophylaxis for deep vein thrombosis), audit tools that use predefined criteria were already available. Where necessary, a tool was developed to audit the process against local guidelines. Opportunistic sampling sessions on the same process were spaced as uniformly as possible across the observation periods and were not repeated within seven days, with the exception of track and trigger scoring (24 hour limit). Additional observations on deep vein thrombosis prophylaxis were made 10 months after completion of the project to measure the sustainability of improvement.
Collection of data on patient safety incidents
Patients admitted under non-general surgical specialties were excluded. We were not able to provide direct observation continuously, mainly because of weekends and holidays. We therefore followed a large convenience sample of patients in both cohorts (63% of patients before the introduction of the intervention and 54% after it). To control for observer bias and ensure consistency, vignettes of all cases were analysed independently by a consultant surgeon (PM), and classification disagreements were resolved by discussion. Our original method38 was modified by setting the standard of care as “full compliance with best practice,” rather than as “full compliance with acceptable practice” on the basis of a consensus view.37
Prospective observation was limited to patients on the surgical emergency unit. Therefore, patients transferred to other wards (23%) were only followed for a portion of their stay in hospital. This category included many of the most seriously ill patients, who were transferred to the intensive care unit, usually via the operating theatre, and then to a specialist ward. Conceivably, events that occurred in these patients during their time on the surgical emergency unit (when often the patient was sickest and most needed expert care) might have influenced outcomes later in the patient journey. We therefore traced all patients who were not discharged directly home and used retrospective case note review, as described in the Harvard Medical Practice Study,39 to identify safety events that occurred after the patient left the surgical emergency unit.
Strategy for change
Lean quality improvement intervention
Our key approach to improving each safety relevant care process was the elimination of waste while trying to redesign a more robust and reliable system. The aim was to study by selecting appropriate metrics the effect on safety of implementing a standard Lean intervention (table 1⇓). Examples of the ways in which we implemented the five principles of Lean were:
The five S’s: reorganising drug cupboards and storage areas
Process mapping: mapping the whole patient pathway and identifying main points of weakness
Error visibility: displaying daily audit results for correct use of thromboembolus deterrent stockings and heparin administration in prophylaxis against deep vein thrombosis
Elimination of waste: reorganisation and labelling of intravenous fluids to ensure they were easily found
PDCA: used for all interventions.
Study measure and intervention used for each safety relevant process studied
The Lean intervention was delivered by a team that comprised an academic expert in Lean (SN), two members of a consultancy specialising in Lean improvement techniques (KM&T ltd), senior (PM) and junior (SK) surgeons, and a human factors expert (KC). All ward staff involved with patient care were invited to educational half day workshops, where they participated in an exercise to identify quality related problems in the ward and learnt basic Lean ideas. This was followed by a five day training event involving 12 members of staff from different disciplines (medical, nursing, pharmacy, housekeeping, and administrative).
Staff were encouraged to address the problems they felt to be most important. Their priorities were not necessarily focused purely on safety, although we shared our objectives and the pre-intervention data on adverse events with them. Front line staff (those whose day to day work involves direct patient contact) were made responsible for the development and implementation of systems redesign. We aimed to make the default (natural) option the correct action. This reduces reliance on individual knowledge or memory, and helps to render the system of work resilient to human failures.
Staff developed about a dozen mini projects in which they applied Lean to ward processes (box 2). These included: redesigning prescription forms to ensure more reliable application of prophylaxis for deep vein thrombosis; new mechanisms for identifying key nursing personnel, to facilitate communication with medical staff; improved organisation of equipment, consumables, and notice boards; removal of redundant and broken equipment; reorganisation of reference documents; clearer communication of medical protocols; and reorganisation and improved restocking protocols of drugs and intravenous fluids. Five of the seven safety relevant care processes measured were addressed by specific Lean projects (table 1⇑).
Box 2: Example of a Lean mini project
The focus and scope of the mini projects were largely driven by concerns raised in workshops with ward staff. An example is the issue of interruptions to nursing care.
Several nurses raised the problem that two main elements made it difficult to complete standard care tasks: exogenous interruptions (often in the form of telephone calls from relatives); and difficulties in efficiently finding supplies and equipment. It is well established in the safety literature that interruptions in task completion have a negative effect on system reliability and safety. The nursing staff were therefore encouraged to devise a simple “low tech” means of measuring this problem. Small cardboard log sheets were created that could be worn during a shift, allowing a rough tally of relevant incidents to be produced.
The collection of this data over several shifts, and presentation and discussion of the findings, ensured engagement and acceptance of an initiative to change working practices. This initiative included changes to the way patients’ relatives were asked to communicate with the ward, adjustments to visiting hours, and the thorough reorganisation of medical supplies. These changes were iterated through PDCA cycles until the level of interruptions had fallen to a level considered satisfactory by the staff. The changes were widely welcomed by the nursing staff and appeared to have a considerable effect on the smooth running of the ward.
The ward manager was given licence to introduce changes to nursing routines without approval from individuals at higher levels of management, who were kept informed of, but took no part in, the intervention. Thus management and senior clinical staff tolerated the study and maintained a “hands off” permissive approach, but provided no active support. Problems generated by interactions with other parts of the hospital—for example, delays in obtaining radiology examinations or access to operating theatres—were beyond the scope of the study.
Effects of change
We observed process compliance before the introduction of the intervention (between 29 April 2007 and 16 August 2007), during which time 969 general surgical emergency patients were admitted, and after completion of the intervention (between 26 April 2008 and 26 August 2008), during which time 1114 surgical emergency patients were admitted. A total of 607 patients in the first cohort and 602 patients in the second were prospectively observed for adverse events and potential adverse events. The diagnostic and demographic profiles of these populations were very similar (table 2⇓). The mean age in the first cohort was 51 years and in the second 53 years, with 42% compared with 48% of cases male, and 32% of cases compared with 33% of cases requiring surgery.
Demographic and clinical details of the pre-intervention and post-intervention cohorts
The pre-intervention compliance with ideal practice for each of the seven processes has previously been reported37 and varied between 23% and 89%. The improvement strategies developed for the five processes that underwent Lean intervention used between one and four PDCA cycles. A brief description of the strategies is given in table 1⇑, and the degree of compliance before and after intervention is shown in table 3⇓.
Compliance with the safety relevant processes studied before and after introduction of the “Lean” intervention
A significant improvement in compliance was observed in all five processes subjected to Lean intervention, but not in the two processes that were not. Improvements from pre-intervention compliance ranged from 28% to 149%. Improvement generally increased with successive PDCA cycles, but not every cycle led to improvement. A delayed audit for sustainability in the deep vein thrombosis prophylaxis project 10 months after the end of the main observation period (January 2009) showed 78% compliance (median compliance during the improvement process 80%; fig 1⇓).
Fig 1 Proportion of patients that received correct deep vein thrombosis (DVT) prophylaxis according to risk level
The 1209 prospectively observed patients experienced 369 safety events on the surgical emergency unit (table 4⇓). Overall, 176 (14.1%) of patients experienced actual harm (adverse events) and 177 (14.6%) were exposed to errors in care but did not experience harm (potential adverse events). A total of 156 (26%) of the 607 patients studied in the pre-intervention period experienced a patient safety incident on the surgical emergency unit, compared with 152 (25%) of the 602 patients followed in the post-intervention period. Despite demonstrable improvements in processes and workplace organisation, there was no evidence of any change in the rate of all events after the Lean intervention, and no significant difference in the ratio of adverse events to potential adverse events before and after intervention (χ2=1.502, P=0.22). The associations of adverse events and potential adverse events with potential modifying were not substantially altered by the intervention: length of stay in hospital and whether surgery was required remained the most important risk factors for patient safety incidents (table 5⇓).
Summary of patient safety incidents and potential patient safety incidents identified before and after introduction of the “Lean” intervention
Association of adverse events and potential adverse events with potential modifying factors before and after introduction of the “Lean” intervention
The most common causes of both adverse events and potential adverse events were delays in management and investigation, followed by inappropriate management (failure to initiate appropriate investigation or treatment and deviation from a routine practice or protocol) and readmission for the same problem (fig 2⇓). Delays in management and investigation that were beyond the control of the project team were responsible for the largest numbers of events. We therefore analysed the residual adverse event and potential adverse event rates before and after intervention excluding these irremediable events, but found no difference between the cohorts (χ2=0.814; P=0.367).
Fig 2 Types of safety incident before and after Lean intervention
A total of 165 (27%) patients in the pre-intervention group and 119 (20%) patients in the post-intervention group were transferred to the intensive care unit, the operating theatre, or to another ward (P=0.000025; table 4⇑). Hospital administrative data showed no changes in bed occupancy or length of stay between the cohorts, which could potentially have accounted for this difference in the proportion of patients transferred. Among the transferred patients, 27 additional patients in the pre-intervention group and nine in the post-intervention group experienced a first event after leaving the surgical emergency unit (χ2=4.837; P=0.028). It was often difficult in these cases to determine whether adverse outcomes had been influenced by events in the surgical emergency unit; however, the total number of patients experiencing any event in the two cohorts was not significantly different (table 4).
Discussion
This study shows that a Lean intervention can achieve major improvements in the reliability of safety relevant care processes, concurring with other findings about quality improvement approaches in healthcare.40 41 Previous reports of using quality improvement to address patient safety have concentrated on single processes, but we were able to show that a multimodal Lean approach is feasible. This is important because single interventions are unlikely to have major effects on overall safety. We also found evidence that improvements could be sustained over considerable periods of time.
The proportion of patients transferred out of the surgical emergency ward fell from 27% to 20% after intervention, suggesting that the Lean intervention significantly improved the efficiency of care. However, we did not detect any significant difference in the rates of adverse events or potential adverse events after the intervention compared with before the intervention. This apparently paradoxical finding requires careful analysis and has implications for the way Lean is implemented within healthcare organisations.
With more than 1200 patients, this represents the largest reported prospective observational study of adverse events. The proportion of patients experiencing actual harm (adverse events), at 14.1%, is slightly higher than that reported in retrospective case note studies from similar healthcare settings.3 37 39 42 43 44 45 This disparity might reflect the ability of prospective observational studies to identify problems not properly documented in case records. We also identified a further 14.2% of patients who were exposed to identical errors in care but did not encounter consequential harm (potential adverse events). The significance of understanding these “near misses” is becoming widely accepted.46 47 48 49 Using a prospective observational study design therefore provides a better representation of actual error frequency than simply measuring rates of iatrogenic harm.
The patient outcome measures we used were not directly linked to any of the safety relevant care processes studied but were intended to identify all adverse safety events from whatever source. More targeted outcome measures would have shown the effects of our improvement exercises more precisely, but would arguably have given us less information about overall patient safety.
Limitations of our study include the uncontrolled design and the study’s vulnerability to observer bias and the Hawthorne effect. A controlled study design would have been superior, especially in protecting against the effects of unidentified secular trends, but the practical barriers to conducting such a study were significant. We did not have manpower to cover two hospitals (that is, a control hospital and a study hospital), for example, and none of the wards in the hospital studied had a profile of safety challenges comparable to that of the surgical emergency unit. Data collectors were aware of the study hypothesis, but most process data were strictly objective, limiting the scope for bias. The identification of an adverse event and a potential adverse event to determine a patient safety incident was reliant on skilled interpretation. In our study, patients were tracked daily by a surgical research fellow and a reviewer independently analysed vignettes of all cases without being informed which population the vignettes were from. In addition, any bias from observer expectation would have been in the direction of improvement after intervention. The Hawthorne effect is unavoidable in this type of intervention study. It is encouraging that one improvement—prophylaxis for deep vein thrombosis—was sustained for months after active observation ceased, without an obvious “drop-off” that might be expected with a pure Hawthorne effect. An additional limitation was our failure to involve patients directly in discussions around priorities—an avoidable error that might have influenced the selection of Lean mini projects away from safety and towards staff concerns with other issues.
Inadequate application of Lean seems an unlikely explanation for its lack of effect on adverse events and potential adverse events given: (a) the involvement of Lean experts in the design and delivery of the initiative; (b) the content and delivery of the intervention; and (c) the substantial improvements achieved in compliance with key process measures. Better explanations come from considering the scope and nature of the intervention. The major causes of adverse events and potential adverse events were delays in investigation and surgical management, which were not addressed by any study initiatives because their correction would have required changes in the operations of other hospital departments. We were therefore not able to address all parts of the “patient journey” or process, a major drawback in attempting to apply Lean. A particularly powerful component of Lean is that quality improvements are driven by front line staff according to their perceptions of what needs to change, but in this study this approach could have led to some important safety issues being neglected in favour of mini projects that more obviously improved ward function overall. For maximal effect, a trade-off might be required between targets based on objective audit of pre-intervention data and those driven by worker empowerment. This balance would have faced us with a more challenging “systems” problem—rather than, for example, a problem with training, staff attitudes, or technology, but might have achieved a more successful outcome.
Finally, despite its size, our study may not have been powerful enough to detect a significant improvement in the rates of adverse events and potential adverse events. However, on the basis of our final event rate ((adverse event+potential adverse event)/admissions), and conventional assumptions of α=0.05 and β=0.8, the study had sufficient power to detect a reduction in events of 25% or more.
Next steps
Lean theory emphasises instilling a culture of continuous improvement via the empowerment of front line staff, and we observed this effect to a striking extent in this study. A longer term and more comprehensive Lean project would have had a better chance of achieving outcome change, especially if initiated, facilitated, and supported extensively by senior management. Our status as a research team without explicit management support for our proposals significantly limited our ability to bring about change, and it is testimony to the power of staff engagement that so much was achieved in this study. Much recent work on quality improvement for safety has focused on whole systems change at the macro level50 51; however, results have been mixed and staff engagement may be more difficult to achieve using this strategy than using our micro level approach. Our experience clearly illustrates that attempts to conduct an isolated Lean intervention in one part of a modern hospital system will encounter serious difficulties because of the complex interactions between all parts of the system. Lean is therefore unlikely to be successful without the senior management support necessary to facilitate change across multiple departments, and without a long term, system-wide commitment.
The striking benefits of Lean on compliance with care processes, and particularly the sustained nature of the changes it bought about, have convinced us of its value in improving quality and safety in surgery, and in hospital care generally. However, engagement of front line staff may not necessarily lead to improvements in safety unless broader systemic issues are considered. Given the almost universally positive reports of Lean as an improvement method, we believe that the dichotomous results of our study illustrate why rigorous, well designed studies are important for quality improvement. It is through such studies that the value and cost effectiveness of interventions in improving safety can be properly evaluated.
Notes
Cite this as: BMJ 2010;341:c5469
Footnotes
Contributors: PMcC had the idea for the study. The study design was developed by PMcC, KC, and SN, with contributions from SK and AH. The observations were conducted by SK, with assistance from YS. The study interventions were conducted by SK with assistance from KC and guidance from SN. Data analysis was led by KC and largely conducted by SK, with guidance by PMcC and other authors. PMcC wrote the first and final drafts of the paper. All authors contributed to redrafting and editing processes. PMcC is the guarantor of the paper.
Funding: This work was funded through a grant from the BUPA Foundation, a UK healthcare research charity, with additional funding from a second charity, the Oxford Medical Research Fund.
Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Ethical permission for the study was obtained from the Milton Keynes Ethics Committee (project ref 4801 05/Q1605/48ORH).
Provenance and peer review: Not commissioned; externally peer reviewed.