Comparative studies and healthcare policy: learning and mislearning across borders

Sources of confusion in cross-border research and policy debate

Why do so many of the cross-border studies of health policy muddle rather than clarify the debates? Consider some examples of inaccurate and misleading reporting on experiences abroad in North America and Western Europe before turning to the reasons why analysts and policy makers have embraced so many untested policy recommendations.

The treatment of Canada's experience with national health insurance is a striking example of uneven and typically inaccurate treatment. In the past three decades, the American media has paid attention intermittingly to Canada's Medicare, but when covering Canada's experience, they do so mostly in very negative terms. The commentators use the vocabulary of ‘socialised medicine,’ when, in fact, Canada's Medicare is social health insurance. (There is no generally accepted definition of ‘social insurance’, but its main characteristics are ­mandatory participation of large population groups or the entire population to create large-scale risk pooling; income-related contributions; no exclusions for certain risks; administration by government or quasi-autonomous agencies; and access to benefits based on needs not on willingness and ability to pay. Private insurance, by contrast, is usually voluntary; administered by private firms; charges premiums for certain groups based on the calculated risk of that pool of insured; and often excludes certain conditions or certain groups that represent high risks and thus high costs to the insurer.) The media depicts a ‘government-run’ system with long waiting lists, inaccessible care, and many Canadian patients and physicians fleeing to the USA to seek care and earn higher incomes unencumbered by government. All of those characterisations are highly exaggerated and largely misleading, as serious scholars have documented. As a second example, many European commentators label American healthcare as a ‘private, consumer-driven’ market system,^3,4 when, in fact, more than 50% of America's $2.7 trillion medical economy is financed by social insurance contributions, public taxes or tax expenditures.

Still different but equally misleading are reports advocating the use of electronic medical records to improve the quality of healthcare and reduce its costs. These reports typically fail to report accurately the actual use and financial gains (or lack of) of that use. They conflate policy intentions and promises with implementation and outcomes (see, for example, reports by the Commonwealth Fund [www.cmwf.org]). European commentators had high expectations for the American Patient Protection and Affordable Care Act (PPACA) in 2010, but they paid little attention to the high barriers to change in America's political institutions, which severely limit the Obama administration's capacity to restrain medical costs or reach the 50 million uninsured Americans. The reform will almost certainly expand coverage, but it will do so in ways that are not easy for Americans to understand – let alone commentators coming new to American politics.

The first source of confusion is the very conceptual location of healthcare and health policy making. Carolyn Tuohy (1993) observed how Canada's two ‘worlds of welfare’ – healthcare versus other social welfare programmes – had diverged by the late 1980s.⁵ Canada's Medicare offers health insurance to all residents and is – similar to the healthcare arrangements in Western Europe – based on principles of solidarity and equity. In contrast, Canada's unemployment and disability programmes are more similar to those of the USA – more parsimonious, with comparatively low levels of benefits and restrictive eligibility criteria. In the 1970s, Canada and several other industrial countries set up independent departments and organisations to administer (public) healthcare, appointing cabinet ministers of separate health departments. Universities followed, with independent schools of public health and health policy departments with their own niche of health policy analysis. Such developments cut the ‘umbilical cord’ between healthcare and the income protection of the modern welfare system, such as old-age pensions, unemployment insurance and sickness payment.⁶ Governments and academics active in the healthcare field increasingly left behind their roots in the domain of social welfare policy. They developed ‘epistemic communities’ – or networks of experts – focused on healthcare organisation, finance and provision and their associated issues of governance. Attention shifted away from the general policy goal of protecting family income against the financial risks of illness or disability. Inspired by the rise of ‘new public management’ ideas, some commentators came to regard healthcare as a conventional industry – and with that came a change in language. Patients became ‘consumers’, department heads and medical directors became managers and chief executive officers, and ­hospital units became profit centres.⁷

A second source of confusion arises from the lack of linguistic clarity. Many of the terms, images and labels used in policy debates and comparative studies are vague – subject to multiple interpretations or outright misleading. They simply fail to describe reality accurately. Such language not only creates barriers for understanding health reforms abroad but also prompts unwarranted generalisations about the applicability of such experience elsewhere. Expressions such as ‘accountable care organisations’ (ACOs) or ‘patient-centred care’, for example, have spread widely in the current health reform literature, but they lack settled meanings. What does the label ‘accountable’ mean? Accountable to whom, why and how? (See also Marmor and Oberlander.)⁸ To take another example, the requirement that all residents of Switzerland and the Netherlands buy health insurance has prompted considerable commentary in the health policy literature. Those universal mandates are often quoted as (successful) examples of ‘consumer-driven healthcare’ that might also serve as models for other countries.^9–12 Yet there is evidence that the Swiss and Dutch healthcare consumers who need healthcare most – patients with chronic illnesses and elderly patients – are the least likely to exert their market power.¹³ The Dutch reform model also travels under the label ‘regulated competition’ between (private) insurers.¹⁵ That label seems to suggest that Dutch insurers rather than patients are driving healthcare change. Still, it is far from clear exactly what those terms convey – ambitions or realities.

There are many other examples of simply misleading labels. Persuasive definitions are commonplace, confusing marketing aspirations with realistic descriptions. For example, the label ‘health maintenance organisation’ (HMO) implies that an organisation so named maintains the health of its patients, but that is, by definition, not on the basis of documented performance. As our examples below suggest, importing terms from the world of marketing is familiar. Do ‘shared-care managers’ actually share their power and incomes with others? Finally, consider primary care as an example of a term with multiple meanings: what, we ask, can be sensibly considered as ‘primary care’? The answer to that question, we found,¹⁶ depends very much on country-specific cultural interpretations and institutional legacies. Most Americans consider the physician they consult regularly as their ‘primary care provider’. On that view, the first regular point of contact in the medical system is primary. In contrast, Dutch and German policy makers consider most of the activities that keep patients out of the hospital as primary care, including home care and non-medical services. In Canadian medical vocabulary, primary care connotes ‘community ­involvement’.

We believe that two other terms – ‘policy’ and ‘healthcare reform’ – deserve more scrutiny than they receive. The word ‘policy’ has four very different meanings: (a) intentions as stated in formal government documents and political papers; (b) the process of implementing announced plans; (c) programme and policy measures actually in place; and (d) common practices of a certain organisation.¹⁷ Studies that ignore these divergent meanings can easily prompt inaccurate conclusions about both the shaping and consequences of health policy in other jurisdictions. Similarly, the term ‘healthcare reform’ can mean a variety of government actions that includes major change as well as ongoing policy adjustments.¹⁸ Many studies loosely define reform as a wide range of government interventions without much effort to present an operational definition – for example, as in Okma and Crivelli,¹³ a substantial shift in financial risk and decision making over the allocation of scarce healthcare resources that allows for meaningful comparison of reform efforts across countries.

Box 1 lists a number of examples of misleading labels. Most of these expressions come from the websites of large foundations – for example, the Commonwealth Fund (www.cmwf.org), Kaiser Family Foundation (http://kff.org) and Robert Wood Johnson Foundation (www.RWJ.org) – while other examples are from the academic or general policy literature. The comments and questions we have added to the box illustrate our general claims about the misleading and faddish language of much of the comparative discussion of health reform.

Box 1.

Frequently used expressions in comparative health policy, with authors’ commentary.

A third major source of confusion is time pressure. Policy makers have to come up, quickly, with solutions for (or at least responses to) policy problems.¹ Looking abroad for new policy ideas is one way to do so, even if those ideas have not been tested properly. Politicians are, after all, in political struggles not academic debates. They have little time to distinguish policy goals and proposals from implemented proposals or policy results. They regularly use cross-national information as ‘warfare ammunition’ for policies that they support on other grounds.² There has been rapid growth of ‘health policy tourism’ to study policy experiences abroad. For example, several delegations of European experts visited California's Kaiser Permanente (KP) in the last few years, as they saw KP as the ultimate role model for ‘integrated care’. Those visits were often very short, sometimes one day only, and most visitors paid little attention to why KP seems to be so successful, ignoring the fact that both the providers contracted by KP and its patients are quite distinct from non-members in several ways. As KP's organisational innovation, which actually developed over decades, emerged through selection by both patients and professionals, the transferability of its model is open to serious study.

Finally, we highlight the limited attention to the importance of understanding the broader political context in which health policy debates take place. Failing to do so commonly leads to unwarranted generalisations or policy recommendations. Healthcare systems across the world do have much in common. On the funding side, they all collect taxes, contributions for social insurance, premiums for private insurance and direct patient payment for medical care. All systems combine, to some degree, public and private provision of healthcare services. The announced policy goals in most industrialised nations and emerging economies are broadly similar – to safeguard access to a good quality of healthcare for all – but there are major differences in how the funding, contracting, payment and management of healthcare takes place. The mix of public and private funding differs, as does the ownership of hospitals and other health facilities. Country-specific cultural and institutional features shape national policy making. Policy makers and policy analysts alike pay too little attention to the ‘transferability’ of policy. The question is under what conditions can we expect measures or policies that worked abroad to also work at home. Richard Rose has argued that the sciences of medicine or economics are ‘landless theories’;¹⁹ they largely ignore the country-specific policy context that enables reforms or creates barriers to change. Political science – the study of the shaping and outcome of public policy – should do better than that.

Different purposes and defensible rules for international comparison

What purpose, then, can be usefully served by cross-national analysis and what are the basic rules of the game? One can enumerate five guiding rules: three positive and two cautionary. The first positive approach of cross-national enquiry allows one to understand one's own circumstances more clearly by comparison. For example, it helps to see the problems, options and evaluations by setting them against those of another context. Knowledge about other countries’ systems offers perspective – not direct lessons. It illuminates subtle differences among nations that are quite similar and sharper differences when applied to very different national arrangements. We might term this ‘illumination without transplantation’ of policy.

Second, cross-national inquiry can help one assess the adequacy of nation-specific accounts of policy development. We can describe that as a defence against explanatory provincialism. Canada, Australia and the USA all debated whether to adopt national health insurance in the decades after the Second World War. All faced heated struggles over this deeply divisive issue, but Australia and Canada had such national programmes by the 1970s, while the USA, with separate programmes for the elderly, the disabled and some groups of the poor, did not. Only careful comparative analysis can separate the fundamental differences from the many developments that accompanied this differentiation. The answer to the above puzzle, it seems, is the distinctive institutional character of American politics and, most particularly, its dispersed structure of authority, which in turn provides many more veto points than the parliamentary structure of the other two federal states.²⁰

Treating cross-national experiences as quasi-experiments is the third rule. In this instance, the more similar the countries compared the more reliable are the inferences one can draw from the quasi-experiment; however, as Rose (1991) cautioned,¹⁹ with that comes limits. The range of options will be narrower as the set of comparative examples is smaller. Relationships that hold over many very different national experiences are likely to be few in number but powerful and thus important. For instance, across the industrial democracies in the past four decades there has been a general trend to mixed systems of payment of physicians. At one time, it would have been possible to categorise nations as devoted to fee-for-service, capitation or salary models, but that is no longer the case. Why that is so and what it suggests is a promising topic for analysis.

The cautionary remarks apply to two misleading approaches to cross-national inquiry. The most familiar is what is called the ‘naïve transplantation’ conception of cross-national learning. The idea is simple: search for best practices and, if found, assume that they can be imported. There are many examples of this in the professional literature, but no social science support for the claim that a practice in one site can be transplanted without adaptation. The opposite vice is what can be termed the ‘fallacy of comparative difference’. The major premise is the contention that any two national sites that differ in any way cannot learn from one another. The factual premise of the ‘we are unique’ approach is that there is always at least one respect in which two nations differ. The conclusion of the syllogism is, therefore, that no policy learning is possible cross-nationally. This is a form of intellectual nihilism, but it is not uncommon. We will end our contribution, then, with a plea for discriminating attention to the purposes and limits of cross-national commentary.

In brief, we advocate five basic rules for any form of international comparative study in health policy. The first is the need to reflect on the very purpose of the undertaking of comparison. The second requires explicit reflection on the location of healthcare in the modern welfare state, considering the public and private responsibilities in terms of the funding, contracting and ownership of healthcare services. The third rule calls for well-defined and operational definitions in policy debates instead of misleading marketing labels. This is particularly relevant to the discussion of ‘models’ to organise and finance healthcare. The fourth rule, linked to the first, is the need to understand the country-specific constellation of dominant values, the political institutions and the role of organised interests in the healthcare domain when assessing the chances of failure or success of given reform proposals.

As a final note, we call attention to some countries that have paid systematic attention to the health policy experience of their neighbours or more distant nations. For example, when Japan considered introducing long-term care insurance for its entire population in the early 1990s, it commissioned experts to travel to countries that already had such schemes. It also invited foreign experts to discuss the variety of options facing Japanese policy makers. Similarly, Taiwan carefully assessed the international range of national health financing models before deciding on its form of national health insurance.¹³ Earlier in the 20th century, many industrialised countries sent delegations of government officials and experts to study the newly introduced social health insurance of Germany. The common element of those examples is that all of those efforts took – and were given –time for careful study and discussion.