Article Text
PDF
Abstract
Background: A series of government initatives in the UK have included strategies to improve the quality of services received by patients, including fundholding, the development of National Service Frameworks, clinical governance, and Personal Medical Services (PMS). PMS represents a new contractual arrangement between government and general practitioners (GPs) which provides new investment in return for more detailed specification of processes and outcomes of care.
Objectives: To evaluate the effects of PMS on the quality of primary mental health care between 1998 and 2001.
Design: Multiple longitudinal case studies. Semi-structured interviews with key staff within practices (GPs, nurses, practice managers) and outside (health authority and primary care group/trust managers).
Sample: Six first wave PMS sites which had specifically planned to improve their mental health care.
Results: Improvements in mental health care were found in some PMS practices and not in others. Five mechanisms associated with successful quality improvement in mental health were identified: clear goals, effective teamwork within the practice, routine use of protocols and audits, additional resources, and effective collaboration with community and secondary care. Sites where these factors were not present struggled to meet their objectives.
Conclusion: The five mechanisms which resulted in improved mental health care were facilitated by the new contractual arrangements in PMS. The new contracts were not a necessary part of these changes, but they enabled sites with an identified interest and motivation to make the changes. The contractual changes were not in themselves sufficient to improve care.
- mental health
- personal medical services
- quality improvement
- general practice
Statistics from Altmetric.com
Primary care mental health policy in the UK has only recently gained prominence. Successive governments have previously stated broad objectives enabling local interpretation and implementation. However, following a series of reports offering general guidance for mental health care in a post-institutional context, in 1999 the Clinical Standards Advisory Group, in focusing on the management of depression, addressed primary as well as secondary care provision. Primary care led services have also been advocated favouring provision of mental health care within the community. In the mid-1990s some general practices used total purchasing and extended fundholding to make improvements in mental health care.1,2
The mental health National Service Framework (NSF)3 introduces an expanded, if often aspirational, focus on mental health inequalities and promotion.1,4 It sets minimum standards which the health and social welfare community are expected to meet, and emphasises protocols and collaboration between health professionals. Mental health is a core part of primary care and primary care trusts (PCTs) must develop and supervise services with performance assessed nationally. The mental health NSF is part of a broader national strategy of quality improvement which includes setting standards, improving care through clinical governance, and monitoring care.5,6
Introduced in 1998 and now a permanent scheme after four waves of piloting, the Personal Medical Services (PMS) contract allows individual practices, groups of practices including PCTs, and/or community trusts to negotiate unique arrangements for service provision.7 PMS contracts encourage approaches to service delivery that promote local solutions to problems such as recruitment and retention, which conventional General Medical Services (GMS) seem unable to address, and provide an opportunity for practices to focus on specific issues. These new types of contract focus on detailed specification of processes and outcomes of care. Such contractual freedom lends itself to developing individualised, boundary crossing services appropriate to mental health care. PMS has become a silent revolution with 29% of English patients now registered with practices working under PMS contracts.
This study examined whether PMS practices with a specific focus on mental health were able to improve their services and the obstacles they encountered. The study was part of a larger evaluation of the effects of PMS on quality of care,8 which was itself part of a multicentre national evaluation of PMS first wave pilots.9
METHODS
Study design
Detailed longitudinal qualitative case studies were conducted in first wave PMS pilots with a specific focus on improving their mental health care. Ethical approval was granted from the North West multicentre research ethics committee.
Sample
First wave PMS sites with an interest in improving the quality of their mental health care were selected. These sites were part of a wider study involving 23 of the 87 first wave PMS pilots chosen purposively, after reviewing all proposals and contracts, for their focus on quality improvement.8 Of these 23 sites, six planned quality improvements in mental health care.
Reflecting the flexibility associated with first wave PMS, a diversity of aims relating to mental health care was encompassed within these six sites. Three had a specific focus on mental health (B, C and E), two incorporated mental health objectives into a wider quality agenda (A and D), and site F addressed mental health as part of a health education programme. Three of the pilots (B, C and D) came from the same geographical area as part of an initiative led by the health authority.
Data collection
Semi-structured interviews were conducted with key stakeholders at each site to assess the organisational effectiveness of the pilots and to examine interviewees’ perceptions, experiences, and understandings of the processes of change. Each site was visited twice, at the start of the pilot period (April 1998) and near the end (March 2001). Interviews were conducted with GPs, practice nurses (PNs), and practice managers (PMs) as well as with health authority (HA) and primary care group/trust managers as appropriate. Practice managers were also sent a questionnaire to record details of the mental health services in their practice in1998 and 2001 (meetings, on-site services, protocols, staff, information for patients). Telephone contact was maintained throughout to negotiate other types of data collection (such as annual reports) and to check on local developments.
Four members of the research team (SC, JR, AS, DW) conducted the interviews. All interviews were taped and supplemented by the interviewer’s notes and materials gathered at sites (such as annual reports and local evaluations of individual sites commissioned by health authorities or local universities).
Eighteen interviews were conducted in 1998 across the six sites (6 GP, 6 PM, 6 HA managers). In 2000/1 18 interviews were conducted across the five remaining sites (4 GP, 5 PM, 2 PN, 5 HA managers, 2 trust managers).
Data analysis
All interviews were taped and fully transcribed. Interviews were then written up as multiple case studies.10 A report was written for each site based on interview transcripts and other relevant documentation (such as annual reports) according to an “a priori” framework in which interviewees’ responses were organised thematically, based on the topics that formed the basis of the interview schedules—that is, strategies to achieve change, perceived barriers. These reports made liberal use of exact quotations. The interviewer highlighted points of conflict or consensus and supplementary materials were drawn on to illustrate or challenge the stakeholders’ assertions. Interview reports were used to assess the context of each pilot in 1998, the mechanisms for change that facilitated or prevented the implementation of objectives, and the outcomes of the pilot as defined by the interviewees themselves.
The analysis draws on the context–mechanism–outcome model of organisations under transformation.11 We sought to understand what had worked, for whom, and in what circumstances by mapping the mechanisms for change at each pilot and looking for common themes. Themes emerging from the data were categorised, classified,12 and compared across categories using the constant comparative method.13 To enhance reliability, reports were analysed independently by two members of the team. The entire team then met to ensure agreement on the interpretation of the findings. Success or failure was evaluated based on an analysis of interview and supplemental data and by reviewing site-specific objectives—for example, whether protocols had been developed.
RESULTS
The experiences of the six sites are summarized in boxes 1–6. Some sites were practice led initiatives (sites B, C and F: boxes 2, 3 and 6) and others were trust led (sites A and E: boxes 1 and 5) or jointly led initiatives (site D: box 4). There were no differences in the experiences of the sites as a result of these starting points. For example, while the most successful practice was practice led (site B; box 2), so were the least successful (sites C and F; boxes 3 and 6).
Box 1 PMS pilot A
Background and aims
A community trust initiated an application to integrate community and primary health care services. People with mental health problems were seen as an important target group. Joining to create a PMS pilot, the primary care practice and community trust agreed on the objective of having dedicated community psychiatric nurse (CPN) time for the practice and a CPN clinic in-house. Other objectives were to implement protocols for referral to secondary care and to develop trust led services for drug and alcohol abuse.
What happened
Named mental health leads (GPs) led the pilot. Agreement was achieved on the dedicated CPN time but space constraints at the surgery prevented the in-house clinic from taking place and meant no on-site CPN services were developed. Referral criteria for the existing drug and alcohol misuse unit were clarified and practice staff reported noticeable improvement in their collaborations with secondary care. Other specific service developments were on hold because of uncertainty about the trust’s future role in mental health services post-PCT. Trust uncertainty about the role of PMS in the light of health service reorganisations hindered progress.
Specific issues
Teamwork was initially strong as a mental health lead was identified, regular meetings were organised, and on-site CPN services were anticipated. However, by 2001 the situation had returned to pre-pilot status, perhaps reflecting frustration at the prevented CPN services in-house. Services increased as new counselling and marriage guidance services were introduced. Protocols and procedures remained underdeveloped throughout the contract but site staff emphasised that they expected to receive a number of protocols via their PCT. Pilot staff felt reasonably focused on patients’ preferences and this focus increased as a new link worker was employed.
Conclusion
Improved mental health care (link worker, counselling) but the pace of change was hindered by space constraints and uncertainty by the trust about how PMS fitted within the PCT structure.
Box 2 PMS pilot B
Background and aims
Practice led application to improve the care of patients with mental health problems by developing protocols, implementing depression guidelines, and collaborating with secondary care providers to develop a drug dependency clinic. This was one of three pilots with a mental health focus functioning in a single health authority—that is, reflecting a health authority agenda to improve the quality of mental health care.
What happened
Additional resources boosted the mental health team. The pilot received funding for a practice counsellor, the upgrading of a junior community psychiatric nurse (CPN), and one third of the cost of a social care manager’s post. The doctors felt the NSF for mental health had “raised their game” and they were convinced that working towards PMS targets gave the whole team an impetus to improve the quality of patient care at a time when the loss of fund holding could have left the practice without a sense of purpose.
Specific issues
The development and implementation of specific protocols was successfully achieved in seven out of nine protocols identified by the end of the pilot. A strong team focus was maintained over the period of the study, starting with a mental health lead (a nurse) and moving on to an explicit multiprofessional team approach. Pilot B succeeded in maintaining good links with the voluntary sector and developed strong relationships with secondary care and their local authority. An on-site clinic, run jointly by a consultant psychiatrist and CPN, was further evidence of productive collaboration with secondary care.
Conclusion
Pilot B had clear aims and targets, received significant growth monies, developed and implemented guidelines and protocols, and forged successful relationships with secondary care providers.
Box 3 PMS pilot C
Background and aims
Practice led application to improve care holistically for all patients and to assess baseline services and patients’ needs, develop a disease register, and set targets. These objectives started and remained vague. Like pilot B, this pilot was one of three with a mental health focus in the same health authority.
What happened
While theoretically this practice had the same monitoring framework as sites B and D, they provided few hard data to the health authority or research team. They updated their recording templates to include depression indicators, set up a register, and developed several new protocols. By 2001 the site had identified a mental health lead (GP). However, staff reported problems obtaining information from secondary care to utilise the disease register. The PMS lead (GP) stated that the mental health NSF derailed their strategy for mental health care.
Specific issues
Teamwork improved modestly including identifying a mental health lead by 2001 and greater perceived clinical leadership. New services were introduced during the first half of the pilot, but later the practice lost its on-site counselling and bereavement clinics and abandoned use of their mental health register. Starting with almost no protocols and procedures in 1998, the practice developed new guidelines for referrals, depression and postnatal depression. In contrast, patient focus weakened and the site reported stopping having written information available.
Conclusion
Limited success reflecting unclear aims and lack of focus. However, sustained teamwork helped the pilot.
Box 4 PMS pilot D
Background and aims
To improve mental health care relating to postnatal depression, antidepressant therapy and bereavement, and to develop monitoring procedures for antidepressant therapy. This pilot had a “PMS+” contract which meant that it could offer services normally funded under GMS as well as services not normally provided under GMS—for example, those from hospital and community health services budgets. Wider objectives included developing an integrated primary health care team (PHCT).
What happened
There were deep divisions in objectives between the practice and the community trust from the start. The pilot started with an attached community psychiatric nurse (CPN) who became the mental health lead. It had specific and well thought out plans. In year 1 they began work developing a protocol to identify and manage depression, a screening tool for use by the PHCT, one information sheet for patients on use of antidepressants, and another to help professionals monitor compliance. READ codes to support data collection were completed and entered onto a standardised software program (EMIS). However, by the end of 1999 progress was halted as the trust changed its mental health strategy and the practice lost its on-site CPN services to a central team. The rest of the practice team were too busy to support the work on their own.
Specific issues
Teamwork deteriorated between 1998 and 2001, reflecting the loss of CPN services and the stopping of regular meetings between the GP and the PHCT; more generally, the loss of on-site provision (counselling and alcohol counselling) hampered the quality of service provision. However, several protocols were developed.
Conclusion
This was a pilot with a planned mental health focus, but external changes limited the practice’s success with using a PMS+ contract to develop more effective teamwork and improve quality of care.
Box 5 PMS pilot E
Background and aims
There was a perceived financial crisis at the practice which faced bankruptcy. While the exact PMS objectives were not clear, they included undertaking a mental health promotion project as part of wider service developments. These involved developing a programme of arts and health, piloting a local arts and health strategy for the local trust, incorporating local volunteers to work in health, and working within the framework of the local mental health unit. The identification and treatment of postnatal depression was a particular focus.
What happened
The site did not achieve any of the service developments outlined above because of problems with finance and staff turnover. Rather, they had to focus on practice infrastructure and delivering basic general medical services. However, they continued to work with an attached CPN.
Specific issues
Teamwork was initially weak but by 2001 the practice had got new on-site access to a CPN and the GP started to meet regularly with the PHCT, and PHCT staff met with secondary care providers to discuss mental health care. The number of services offered was comparable to other pilots and remained constant during the period of the study. No use of protocols and procedures was reported, suggesting no progress in developing protocols for postnatal depression.
Conclusion
There was a lack of shared vision about the precise links between arts and health and education, and the practice suffered from financial problems and poor premises. However, improved teamwork and trust support enabled the practice to retain its PMS status.
Box 6 PMS pilot F
Background and aims
This practice led application sought to improve patients’ mental health as part of a wider strategy focusing on health education and access to primary care. There was a marginal mental health focus which centred on appointing a family health coordinator and a link worker, promoting exercise to reduce depression, and extending services to adolescents that included psychological counselling.
What happened
The pilot focused on service developments for families with children under 5 (drop-in clinic), ethnic minorities (health education group), and the elderly (lunch club). These initiatives had only a limited success. No real attention was given to developing mental health services. Very little infrastructure was in place to ensure quality of mental health care in 1998. No protocols and procedures existed in 1998 and none were developed. The pilot had withdrawn by 2001.
Conclusion
The pilot was always ambivalent about its PMS status and eventually pulled out.
We found no single effect of the new contractual arrangements (“PMS effect”) in mental health but many effects in individual practices which reported diverse experiences even when contracts had been negotiated with the same health authority.
In general, the sites can be categorised into one that successfully met its mental health objectives (site B, box 2), one that had a mixed but mostly positive experience (site A, box 1), two that had a mixed but mostly negative experience (sites C and E, boxes 3 and 5), and two that essentially failed to meet their objectives (sites D and F, boxes 4 and 6).
The pilots that successfully met all or many of their objectives (sites A and B) were characterised by similar positive change mechanisms (box 7). Pilot B, which had been a fund holding practice with a high quality of care at the outset, was the most successful in meeting its goals (box 2). It used its new contractual arrangements as a motivator and worked to improve quality of care in line with the mental health NSF. In pilot A (box 1), which was initiated by the community trust, effective relations between the primary and secondary care sector were crucial to the initial success although uncertainty over roles produced by the introduction of primary care groups hindered subsequent progress.
Box 7 Mechanisms associated with positive change
-
Clear aims and vision shared between practice staff, community trust partners, and the health authority.
-
Good teamwork.
-
Positive collaboration with health authority, hospitals, and community based secondary care.
-
Additional financial resources.
-
Development of protocols.
The sites that encountered difficulties in improving mental health care were characterised by a recurring, if only partly overlapping, set of problems. Some had unclear objectives, weak commitment, or mental health objectives that were part of a wider strategy rather than a specific priority. Others described poor teamwork, inadequate premises, ineffective or unsupportive collaboration with health authorities, secondary care and/or community trusts, and insufficient financial resources. Any one of these could undermine an otherwise promising experience (see, for example, site D in box 4).
The case studies highlighted the importance of five particular factors that acted as mechanisms for successful or unsuccessful quality improvement:
-
effective collaboration with community and secondary care;
-
effective teamwork within the practice;
-
routine use of protocols and audits;
-
clear objectives;
-
additional financial resources.
Effective collaboration with community and secondary care was important in all six case studies. Unsuccessful external collaboration had a consistently negative effect. For example, site A (box 1) set up a mental health register but by 2001 it was no longer being used as the practice found it too difficult to access information from secondary care. Similarly, site D (box 4) had their plans aborted because the local trust altered its mental health strategy. While PMS was meant to give practices freedom, success was dependent upon external collaboration.
Building team based care, either within the practice or between the practice and external partners (such as health authorities), was a central ambition of three of the PMS pilots in this sample (sites C, D and E).
Protocol development was integral to the quality goals of three sites (B, C and D) and site A was waiting for its PCT to circulate protocols.
Sites with clearer objectives at baseline met more of their mental health objectives (site B) than those with unclear aims or lack of a shared vision (sites C and E). Until 1999, the clear objectives at site D drove numerous successes but then factors outside the control of the pilot intervened, undermining subsequent quality improvements (box 4). Site F was a practice with unclear aims and withdrew midway through their contract (box 6).
Additional financial resources played an important role in facilitating (site B) or preventing quality improvements (sites D and E).
While each of these mechanisms was associated with change, none of them in isolation was sufficient to secure overall quality improvement. For example, site D had clear objectives but was stymied by its unsuccessful collaboration with its local trust (box 4). Successful quality improvement needed an environment where all (site B) or most (site A) of these factors were present. Moreover, PMS acted as an enabling mechanism for change with practices reporting that they “could have but wouldn’t have” made changes without PMS pilot status, which focused their attention upon specific goals.
DISCUSSION
Summary of main findings
The case studies showed that a focus on changing mental health care was not sufficient to create quality improvements, with four of the six sites not having a positive experience. Five mechanisms were associated with successful quality improvement in mental health. These were clear goals, effective teamwork within the practice, routine use of protocols and audits, additional resources, and effective collaboration with community and secondary care. This suggests that the new contractual arrangements were an enabling mechanism for practices with an identified interest and motivation to change their clinical effectiveness, but not an automatic catalyst for quality improvement.
Limitations of the study
The case studies were based on practice level services and the experiences of practice staff but did not include the opinions and experiences of patients with mental health problems, except by second hand reporting. While patients’ experiences are important, this does not detract from the importance of the views of those stakeholders involved on a day to day basis with attempting to implement quality improvement in primary care (GPs, nurses, managers). Moreover, while we administered questionnaires to practice managers and reviewed annual reports and local evaluations of individual sites by universities or health authorities as part of the case studies (detailed in boxes 1–6), no other corroborating evidence was collected. Supplemental data could also be obtained from, for example, patient evaluations, analysis of prescribing indicators, or hospital admission and referral rates.
Implications for policy and practice
While each of the five change mechanisms was facilitated by the new contractual arrangements (PMS), the contracts were not a necessary part of this change nor were they in themselves sufficient to improve the quality of mental health care. This is because the mechanisms identified are associated—and have resonance—with quality improvement strategies generally rather than the contractual basis of the primary care providers in this evaluation. For example, clinical effectiveness in primary mental health care depends on joint working between professionals1 because of the need to engage with numerous social and healthcare providers. These case studies show that successful collaboration across the interface between primary and secondary care was a positive catalyst in two sites (B and E) and, conversely, unsuccessful collaboration inhibited success in two others (A and D). Clear aims also facilitate successful quality improvement14 as shown by site B, whereas sites C and E struggled from the start because of poorly defined aims, and site F withdrew.
Guidelines have been the prominent quality intervention strategy for the management of mental health problems in primary care—for example, for depression.15 However, there is limited evidence of their effectiveness16 and many GPs do not use them.17 The NSF for mental health advocates the use of protocols. Protocols were also championed by four of the six sites in this study, although in only one site were protocols developed within a climate of overall successful quality improvement (site B). This supports the literature that suggests that protocols alone are not sufficient to generate quality improvement.18–20
Effective teamwork and collaboration are essential for good service delivery, both generally21–23 and for mental health care.24 In sites where teamwork was perceived to have been successful (sites B, C, E), this aided their capacity to meet their objectives but, where it had been negative or weak, objectives were less clearly met (sites A and D). Professional and interpersonal issues such as professional boundary disputes, role conflicts and tensions concerning power, autonomy, and control have been found to undermine good team working.25
The fact that some of these PMS sites—which had a specific focus on improving mental health care and received additional resources—experienced barriers and, in most cases, failed to meet their objectives has implications for improving mental health care in practices with no such agenda—notably, in GMS practices. There is already variation between primary care providers,1,16 uncertainty about how to address the mental health agenda,26 and limited measures available to assess the quality of mental health services at the primary care level.1,27–29
The challenge for the implementers of the standards stipulated in the NSF, as well as quality improvements in primary care mental health services generally, is to ensure that the necessary primary care mental health infrastructure is in place to meet the NSF standards. In addition, realistic expectations are vital as some PMS sites in this study set ambitious objectives which could not always be completed and others found it difficult to sustain their enthusiasm, partly due to deteriorating collaboration with external agencies, especially community or mental health trusts (for example, sites C and F). NSFs which set out aspirational aims and standards but without a plan for organisational development, such as the mental health NSF, perhaps risk achieving fragmented quality improvement.
While the NSF for mental health was seen as “derailing” the efforts of one site (site C), it “raised the game” of another (site B). The role of the NSF within these two sites is, however, probably incidental to the wider context of each practice’s PMS pilot experience. The site that felt enabled by the NSF had clear aims and experienced successful quality improvements for mental health (site B), whereas the site that felt inhibited had unclear aims and experienced only limited success (site C). This suggests that NSF implementation may be as contingent upon internal organisational factors as external ones such as PCT strategies.
Key messages
-
Case studies in six general practices in England with a first wave Personal Medical Services (PMS) contract were performed to determine the effect of PMS on improving mental health care.
-
Only one practice successfully met its mental health objectives and another had a mixed but mostly positive experience. Two practices had mostly negative experiences and the remaining two failed to meet their objectives.
-
The five mechanisms associated with successful quality improvement were clear goals, effective teamwork within the practice, routine use of protocols and audits, additional resources, and effective collaboration with community and secondary care. Sites where these factors were not present struggled to meet their objectives.
-
The new contractual arrangements provided an enabling mechanism for practices with an identified interest and the motivation to make changes in their clinical effectiveness, but were not in themselves sufficient to improve the quality of mental health care.
The quality improvement literature suggests that multiple systems approaches, which focus on all relevant stakeholders and organisations, are most effective in implementing quality improvement.14,30,31 This study also shows that quality improvement in mental health care depends upon contextual factors both internal to the organisation under change (aim, teamwork) and outside (effective collaboration, resources).
In conclusion, there was no single effect of the new PMS contractual arrangements on the quality of mental health care. However, we identified five factors that acted as enabling mechanisms for successful or unsuccessful attempts to improve mental health care in six first wave PMS sites. Sites where these factors were not present struggled to meet their objectives. The new contractual arrangements were an enabling mechanism for pilots with an identified interest and the motivation to make changes, but were not in themselves sufficient to improve the quality of mental health care.
Acknowledgments
The authors would like to thank members of staff in all the PMS and GMS practices which took part in the study as well as staff in the relevant health authorities and primary care groups/trusts, Linda Gask, Sally Richards, Anne Rogers and, in particular, Diane Coulson for their contribution to this project.