ABSTRACT
‘Sustainable value’ considers patient and population outcomes against environmental, social and economic costs or impacts, providing a framework for driving sustainable improvements in healthcare for current and future generations. Measuring the impact of a quality improvement initiative on sustainable value is a new endeavour. For this to be both meaningful and useful, we must balance academic rigour (using a reproducible methodology to capture the most relevant and important impacts) against pragmatism (working within the constraints of available time and data). Using case studies, we discuss how the different variables of sustainable value may be measured in practice.
Introduction
We have argued (in Sustainability in Quality Improvement: redefining value in this issue)1 for an expansion of the definition of value in healthcare to consider patient and population outcomes against a ‘triple bottom line’ of environmental, social and economic costs or impacts. By adopting this more holistic definition of value and using it to drive quality improvement efforts, we can support the evolution of ethical health services, which contribute positively to building health in communities and which can be sustained to provide high-quality care to patients not just of today but of the future.
It is increasingly well recognised that, due to the complex interplay of contextual and methodological factors, quality improvement initiatives do not necessarily produce the benefits that are intended.2 In response, the academic discipline of improvement research is now being developed, aiming to ‘understand, help develop, and evaluate improvements to health care practices and policies’.3 This presents a timely opportunity to embed sustainable value as a guiding framework for the future development of healthcare improvement. But how might one ‘evaluate improvements’ from a sustainability perspective?
In this article we aim to provide a starting point for developing sustainability evaluation in the context of quality improvement. Beginning with the sustainable value equation (Fig 1), we discuss how the different variables may be measured in practice.
Sustainable value in healthcare. Reproduced with permission from Mortimer et al, 2018.1
Measuring impact / measuring value
Improvement projects are often planned using a logic model or logical framework,4 by which intended outcomes, outputs and activities are developed in support of a stated goal(s), along with objective indicators to allow evaluation at each level. Measuring impact as discussed here is distinct from the use of such process-based indicators to measure progress within a project, referring instead to the original goal of the improvement, which we have argued should be ‘to deliver maximum health gain with minimum financial cost and harmful environmental impacts, while adding social value at every opportunity’. This represents a significant broadening of focus for current quality improvement initiatives, many of which are driven by an exclusively clinical focus for a defined sub-population (eg reducing morbidity and mortality from X intervention in Y patient group), ignoring population-level outcomes and value from resources. Where resource use is currently considered, this is almost universally restricted to the financial cost of providing care.
We propose that potential impacts (positive and negative) be considered for each of the five variables in the sustainable value equation and pragmatic outcome measures identified to capture the most important impacts in each category:
patient health outcomes
population health outcomes
environmental impacts
social impacts
financial impacts.
In this multidimensional approach, we do not intend that all the outcome measures be converted into a common unit of measurement to enable the equation to be resolved as a single numerical value. Rather, the intention is to make visible the true costs and benefits at a system-wide level, to inform and drive improvements in each area.
Importantly, since value streams follow patient pathways and may cross organisational boundaries, when measuring value, we should do our best to look across the whole system, not just at individual steps within the patient pathway.
Measuring patient and population outcomes
Systematic measurement of comprehensive, patient-centred outcomes is a powerful tool for driving improvements in care. Outcomes are the results of care on the health of patients and populations and are distinct from interventions or processes involved in delivering care (which are more often measured, being easier to measure).6 The identification of standard outcome indicators for different medical conditions and patient groups is an ongoing endeavour.7,8 In a quality improvement project, a pragmatic approach is to establish a threshold of good quality based, where possible, on internationally agreed outcome measures. Following the improvement intervention, the service should meet or exceed the defined ‘good quality’ threshold, and any known harms / adverse effects should subsequently be subtracted to demonstrate a net effect.
Service provision should consider an entire population as well as the individual patient, aiming to meet the needs of all groups. Even an excellent service, if it only helps a few individuals, provides limited and unequal benefit. Therefore, data on relevant population-level outcomes should be obtained to provide context to the health outcomes achieved in the defined patient group, highlighting any unmet need.
Measuring financial costs
Health economics is an academic discipline in its own right with an entire published literature of its own. Many larger quality improvement programmes can call on health economists to help with economic evaluation of the costs of providing care as well as the economic impacts of ill-health or adverse events (although the application of health economics in this context is itself relatively immature).9 However, for those without access to academic support, guidance for calculating cost-benefit analysis can be found elsewhere.10
Measuring environmental costs: carbon foot-printing
Although not the only environmental consideration, climate change is recognised as the gravest and most urgent environmental threat to human health. A carbon footprint is the sum of greenhouse gas (GHG) emissions (which drive climate change) attributable to a given process, and is widely used as a measure of environmental impact.
The NHS is committed to meet carbon reduction targets of 34% by 2020 and 80% by 2050 (on a 1990 baseline), in line with the UK Climate Change Act of 2008. The impact of a quality improvement project on the NHS carbon footprint can be estimated by converting data on travel, consultations, hospital admissions or other activity into kilograms of carbon dioxide equivalents (CO2e) using GHG ‘emissions factors’. (Generic GHG emissions factors are published by the UK government, or alternatively, healthcare-specific guidance is available from the NHS Sustainable Development Unit.)
Case study 1 (Box 1) illustrates the measurement of financial and environmental impacts from a quality improvement project.
Case study 1: estimating financial and carbon impacts of a primary care-based QI project
Measuring social impacts
There is no standardised methodology for measuring social impacts in the context of healthcare. Health services can influence the social circumstances of patients, carers, dependents, staff and communities both local and distant – such as those affected by activities in the supply chain. Impacts may also be diverse: affecting housing status, employment status or conditions, even social inclusion and relationships.13 Although there is not a simple formula, it is valuable to consider the range of people affected by a care system, the size and nature of any potential impacts from a service change and, where possible, their importance to the people concerned. Having identified impacts that are likely to be most relevant in a given care system, public health colleagues can often provide help in determining how these may be measured, and in accessing any existing available data.
Table 1 lists key themes of social sustainability which can help to provide a structure when considering social impacts. Case study 2 (Box 2) illustrates a pragmatic approach to measuring the social impacts – in this case, of an existing service – on staff, patients and carers.
Key themes of social sustainabilitya
Case study 2: measuring social impact of a dementia service
Discussion
Measuring the impact of improvement initiatives is central to the emerging discipline of improvement science, enabling effective strategies to be distinguished from ineffective or harmful ones.4 It is critical that impact measures be aligned to the overall goal of quality and value improvement. Current approaches tend to focus on clinical impacts chosen for their relevance to (and amenability to change by) specific services, which may not capture those impacts most important to patients, and may not reflect the care provided by the care system as a whole to all the people in need in a population. Meanwhile, the resource implications of improvement are poorly studied and almost universally limited to financial resources, ignoring environmental and social resource use. If outcomes are defined too narrowly, and the resource implications of change are not considered, then opportunities to improve the use of resources for patient and population benefit will continue to be missed.
We believe that adopting sustainable value as a guiding framework for improvement science will foster a more holistic, resource-based approach in healthcare improvement and inform the development of initiatives that build towards a sustainable future.
There is a need for balance between measuring what is important and overburdening those attempting to make improvements with data collection and analysis. We believe that giving thought to each variable in the sustainable value equation and how it could be measured and improved could become part of every improvement project, while full sustainability impact analysis will be research led.
Over time, as the framework is established in practice, we would expect that the proposed methodologies become more standardised for the different types of service and quality improvement intervention, reducing the burden on improvement teams and increasing their capacity to conduct meaningful impact evaluation.
Conclusion
The redefinition of value in healthcare to include environmental and social dimensions is necessary in order to support the creation of a more ethical, sustainable health service. To be meaningful, the new definition needs also to be measurable. We have provided guidance on how the different elements of sustainable value may be captured and quantified in the context of a quality improvement initiative. The multidimensional approach to measuring value provides a consistent yet flexible framework to highlight important impacts (such as effects on staff working conditions or patient time and travel) that have too often been disregarded – to the detriment of patients, services and communities.
Author contributions
FM was the main author and led on conception, design, data analysis and drafting. JI contributed to design and drafting. CK contributed to data acquisition and analysis, drafting and critical review. MP contribed to data acquisition and analysis, drafting and critical review. EV contributed to design and critical review. All authors gave final approval of the manuscript and agreement to accountability.
- © Royal College of Physicians 2018. All rights reserved.
Jump to section
Related Articles
Cited By...
- No citing articles found.