Including a wider range of values in healthcare policy: how can public value evaluation help?

Values in healthcare governance

Different modes of healthcare governance (hierarchy, competition and collaboration) foreground and facilitate different values. Each implies different practices and is embedded in a broader system of ideas about how to administer and organise healthcare: hierarchy in bureaucracies, competition in markets and collaboration in networks.

In a hierarchically organised bureaucratic welfare state, importance is given to assuring citizens' rights to access services. Even those aspects of bureaucracy that are often seen as downsides (such as inertia and ‘red tape’) are valued for the way that they protect citizens' rights from the vagaries of political interests and the subjective decisions of staff.⁹ Bureaucracies value political and professional knowledge. Politicians are assumed to represent citizens interests and professionals are trained to take citizens' needs into account. Authority sits with government and decisions are implemented from the top down through hierarchical layers of distributed responsibilities. Service change and quality improvement is initiated through top-down strategic planning and new professional standards.¹⁰ Hierarchy has been a feature of the NHS in England since its inception, and even when national policy has introduced different governance mechanisms (such as competition or collaboration), it has remained as an important cultural template that influences what happens and what is done at different levels.¹¹ The persistence of hierarchy is a feature of tax-funded systems (such as the NHS) where citizens and their representatives ‘will, in one way or another, sooner or later, insist on accountability on the part of those who act in their name using resources appropriated from them.’¹²

The widespread use of competition in healthcare governance dates from the 1980s, prompted by concerns over public expenditure.¹³ Competition is the core governance mechanism of markets and is valued for its potential to secure efficiency. Markets value management knowledge, particularly as it relates to the practices of the private sector. Politicians act as ‘meta-governors’ ensuring the conditions are in place to support competition. In theory, service change and improvement are driven by the choices made by citizens between different services. In practice, the nature of healthcare markets mean that insurers or commissioners tend to act as a ‘proxy consumer’. Elements of competition have featured in national policies in the NHS in England since 1990, becoming a statutory obligation in the Health and Social Care Act 2012.¹⁴ This obligation was removed in the Health and Care Act 2022, as collaboration replaced competition as a key governance mechanism in national healthcare policy.¹⁵

Integrated care systems (ICSs), formally established in the NHS in England in July 2022, are an example of networks, in this case, locality-based networks that bring together health and social care partners (including commissioners, providers, and local authorities) to collectively plan health and care services to meet the needs of the population. A key objective of ICSs is to reduce the long-standing problem of fragmentated care experienced by patients. The key governing mechanism of networks is collaboration. Collaboration is often seen as beneficial to durable public-sector innovation in the way that it supports the incorporation of diverse perspectives, essential for creativity and real-world effectiveness of programmes and interventions.³ Collaborative networks value the knowledge of a range of stakeholders, which in healthcare may include different health and social care providers, other public sector agencies, private sector organisations, and patients and citizens. As with competition, the role of politicians in collaborative networks is as ‘meta-governors’ ensuring the necessary conditions are in place to support collaboration. The role of professionals is as experts, working collaboratively with other stakeholders to create new policies and solutions to complex problems. For collaboration to be successful, all relevant stakeholders need to be included, there needs to be trust between different stakeholders, and an appreciation of different interests and values. Importantly, there needs to be commitment to the process of good faith bargaining for mutual gain and consensus-orientated decision making.¹⁶

ICSs have, however, been introduced into local settings that still bear the remnants of earlier modes of governance, namely a previous policy commitment to the use of healthcare markets and enduring forms of hierarchy. Historical shifts in healthcare governance have left behind layers of practices and ideas that have accumulated over time and become ‘sedimented’, creating a complex mix of governance mechanisms and forms of valuation.¹¹ The effects of policies and programmes are shaped by how these different mechanisms interact in local settings; for example, networks are often introduced to engender local flexibility, innovation and pace of change. The effect of hierarchies on networks, however, is to reduce their autonomy, distinctiveness and effectiveness.¹⁷

The relative influence of different modes of governance on activities varies across localities, dependent, in part, on local culture and traditions. In some localities, the NHS hierarchy continues to be the main influence on the priorities of healthcare managers, even in the absence of bureaucratic structures.¹¹ In these instances, the NHS hierarchy continues to work through hard forms of power (such as budgets and central targets) but also through ‘soft power’ (such as forms of data collection, analysis and evaluation). In the example of integrated care networks, research has shown how the requirements of data collection and analysis direct the focus of local staff to central government priorities (such as activity and cost savings) rather than network objectives (such as improved patient experience and care coordination).¹¹ Evaluation can work the same way, foregrounding and reinforcing some values and excluding others, as captured in the expression ‘what's measured is what matters’.

Public value evaluation

Different governing arrangements need different approaches to evaluation. New approaches are needed to support the objectives of ICSs for inclusive decision making and durable innovation, and as a check against the tendency for the priorities of the NHS hierarchy to displace local plans. Public value evaluation is one approach that can be used to include a wider range of values in analysis. Public value evaluation is based on the scholarship of John Moore who produced a policy implementation framework based on a study of the real-world work of public sector managers.¹⁸ In this framework, public sector organisations produce ‘public value’, a concept analogous to that of shareholder value in the private sector. This is done within the limits set by the authorising environment and organisational capacity (Fig 1).¹⁸

• Download figure
• Open in new tab
• Download powerpoint

Fig 1.

The public value implementation triangle.

Fig 1 draws our attention to key analytical components of public sector policies and programmes.

• They create something that is valued by stakeholders. As the inevitable product of politics, this is not an absolute standard but necessarily historically and geographically emergent.

• The authorising context for management strategy requires that this is seen to be legitimate and politically sustainable, ie able to attract sufficient ongoing support and resources from political and other stakeholders.

• Management strategies must be operationally and administratively feasible.

The framework is empirically grounded and has strong face validity with public sector managers.¹⁹ This suggests that it is a potentially useful evaluation framework. To the extent that it foregrounds important features of the empirical reality of policy implementation, it is especially useful for identifying contingencies that can explain the effects of any given policy and programme. The concept of public value has been interpreted in different ways, so researchers must be clear about how they are using the term.¹⁹ What follows is my own interpretation of how the implementation triangle can be used in policy and programme evaluation.

Orthodox approaches to policy and programme evaluation in healthcare often proceed by identifying the objectives or eliciting the ‘programme theory’ of a policy.²⁰ In contrast, a public value evaluation begins by exploring what is important to relevant stakeholders. When I use the term ‘stakeholders’, I am referring to all social groups who have a stake in the policy or programme. In healthcare contexts, this may include policy makers, managers, clinicians, patients and communities. What I am suggesting is an approach to evaluation that asks the following questions.

• What do different stakeholders value? And what are the effects of the policy or programme for these values?

• What outcomes or attributes of the policy or programme do different stakeholders value?

The proposed approach, therefore, combines a ‘bottom up’ and ‘top down’ perspective. The ‘bottom up’ perspective considers what is important to different stakeholders, as a backdrop for analysis of the effects of the policy or programme. This broad lens can capture the full range of effects: benefits and disbenefits, both intended and unintended, for participants and non-participants. The ‘top-down’ perspective considers how a specific policy or programme is experienced, interpreted and adopted by different stakeholders, which is essential for understanding and explaining effects. This perspective considers both processes and outcomes. This is important because care processes are often highly valued by people, especially patients and families.²¹

A key feature of the approach I am proposing is that it includes not just the benefits to individuals but also broader social and collective benefits. It also includes more intangible benefits, such as social capital. Social capital has been defined as ‘features of social organization, such as networks, norms, and trust, that facilitate coordination and cooperation for mutual benefit.’²² Some notion of social benefits is often included in orthodox approaches to evaluation. In the Kellogg logic model, for example, it is included under long-term ‘impacts’. Nevertheless, in practice, the focus is predominantly on individual outcomes, and while the causal pathway to longer-term impacts is often specified, these are not themselves measured.²³ Public value evaluation is particularly well-suited to public sector policies and programmes where there are distinctly public values, such as fairness, justice, equality and the production of public goods.²³

An additional benefit of using Moore's implementation triangle as a framework for evaluation is that it identifies important constraints on policy implementation, namely features of the authorising environment and organisational capacity. This avoids the problem of ‘thin’ analyses comprising lists of well known ‘barriers and facilitators’. Instead, analysis focuses on how benefits are produced in public sector settings. Here social science-based analysis can explain how and why programme effects are produced and the necessary conditions for success.^24,25 While this is often touched on in orthodox approaches to evaluation, especially input evaluation and process evaluation, there is often insufficient attention to the capacity of the delivery organisation to produce public value.²³

What different stakeholders value in any given setting is not absolute, but historically and geographically emergent. Public value evaluation strategies are, therefore, flexible to fit the particular circumstances. There are lots of different methods to choose from when designing an evaluation strategy, with a ‘mixed methods’ approach likely to support a more complete picture of what different stakeholders value and the range of policy and programme effects. Qualitative methods (such as individual or group interviews) could be used for initial exploration of stakeholder values. Arts-based approaches or appreciative inquiry may be useful here. I have recently used vignettes (scenarios) to open conversations with patients about what they value about NHS dentistry.²⁶ Systematically collecting qualitative data could also be used to capture programme effects, for example, by using the Community Capitals Framework, which identifies seven forms of capital (cultural, human, social, political, financial, built and natural), or Ripple Effect Mapping, a form of mind mapping that helps a group reflect on the broader effects of a programme.^23,27 There are also an increasing choice of innovative quantitative methods and forms of economic evaluation (such as social network analysis and sophisticated cost-benefit analysis).

The broader lens afforded by public value evaluation has many benefits. Importantly, it can be used to include the values of stakeholders previously excluded from analysis; for example, the wellbeing of healthcare staff, what is important to hospital inpatients, what communities value about healthcare facilities, and what is important to children and young people who are the target of policies and programmes.^7,28–30 The contribution is in broadening and deepening analysis; for example, it might include the social and human needs of workers, the importance of good working relationships and meaningful work. By mapping the range of stakeholders, and what is important to them, the evaluation is more relevant to stakeholders. There will be tensions between different values, these are inevitable in decisions about public services. It is, nevertheless, important to make a wider range of values visible to enhance conversations and support collaborative processes.