Abstract
Strengthening clinical audit is crucial for improving the quality of healthcare provision. The West Midlands Rheumatology Service and Training Committee coordinates an innovative programme of regional audits and the experience of rheumatology healthcare professionals involved was surveyed. This was a questionnaire-based study in which respondents rated statements relating to regional audit on Likert scales. Out of 105 staff, 70 replied. There was consensus that results of regional audit have been robust, valid and reliable; regional audits benefit patients and units; provide educational opportunities for specialist registrars (SpRs); and are more efficient than local audit by allowing comparison between units. Opinion was divided about how well informed respondents were and how effective they are at closing the audit loop. Many units reported changes in practice. Regional audit is widely perceived to be a valuable clinical governance tool supporting significant changes to clinical practice, and an excellent training opportunity for SpRs. Recommendations for a successful regional audit scheme are described in this article.
Introduction
It is apparent that the strengthening of clinical audit will play a crucial role in current plans for improving healthcare provision. Increased emphasis on improving quality of patient services has been highlighted in the recent Darzi Report and the National Clinical Audit Advisory Group has been established to provide a coherent programme of audit activities and to support national and local audit.1,2 Evidence of participation in quality, multicentre audit is likely to play a role in staff appraisal including consultant revalidation.3
National audit has been conducted in the UK since 1998 in specialties with National Service Frameworks (NSFs), as developed by the Clinical Evaluation and Effectiveness Unit (CEEu) at the Royal College of Physicians. It overcomes many of the well-cited problems with local audit including:
projects often being small scale with too few patients to produce meaningful results
results being retained locally so there is no external pressure for change
the audit loop often not closed with repeat audit.4
National audits have led to significant improvements in healthcare and are likely to play an increasing role in the development of clinical audit.5 However, these national audits incur significant costs in reporting and analysing data, currently remain limited to specialties with a relevant NSF and, importantly, will not give junior doctors the experience and educational opportunities afforded from designing and leading an audit project themselves.
Not all multicentre audit is at national level; several regional multicentre audits have been conducted and published.6–8 In 2000, the West Midlands Rheumatology Service and Training Committee (WMRSTC) started an innovative, voluntary, annual programme of regional audits, designed and organised with specialist registrar (SpR) involvement, based on national specialty guidelines. The typical methodology used is described in Box 1.
An example method of a rheumatology regional audit in the West Midlands.
Using this methodology, six regional audits were completed by the time of this study with no external funding. Each involved approximately 2,000 patients across 10 to 13 units (Table 1).9–14
Summary of patient and unit numbers involved in the West Midlands rheumatology regional audits to date.
The authors previously conducted a qualitative study with a small sample of healthcare professionals to evaluate this innovative regional audit programme. Findings from this work suggest that regional audit has specific strengths in the clinical governance armamentarium.15 Regional audit was perceived to:
be an efficient use of resources and expertise
useful for specific topics with smaller patient numbers per unit
allow benchmarking against other local units
be a potentially powerful political tool
provide training opportunities in generic skills for trainees
develop collaborative regional networks.
In addition, regional programmes were reported to overcome barriers to audit, such as lack of time and resources, and to incorporate factors that promote successful audit, such as sound leadership and having a supportive environment.15,16
This current study was targeted at all healthcare professionals involved in the West Midlands rheumatology regional audit scheme. Informed by previous qualitative work, the study's aims were to identify views on the regional audit programme, its advantages and challenges, to measure recall of previous audits and whether respondents recollected the outcomes, to assess the impact of previous regional audits on local clinical practice and finally to produce a list of recommendations for successful regional audit, informed by stakeholder opinion, which may be of use not only for further development of this audit programme, but also to other regions and specialties.
Methods
Procedure
Research ethics committee approval was gained. A questionnaire was designed which collected demographic information and on which respondents were asked to rate various statements relating to regional audit on a Likert scale, ranging from strongly agree to strongly disagree. Questions were positively and negatively phrased to minimise acquiescence. Recall of previous regional audits, whether they had participated in the audit and whether they had received the results or heard them presented was ascertained. Details of any changes in clinical practice which had occurred as a consequence of each previous regional audit, such as a change to local protocols, whether the results had informed policy discussions with the relevant management and/or a business case for increased resources, and whether a local re-audit had been performed, were also collected. Lastly there was the opportunity for free text comments.
Participants
The questionnaire was posted to all rheumatology consultants, SpRs and clinical nurse specialists (CNSs) in the West Midlands who had been involved in at least one of the previous regional audits. Repeat mailings were used to optimise the response rate.
Results
The response rate was 70/105 (67%), with replies from all 13 units. Of the 70 respondents, 39 were consultants, 15 were SpRs and 16 were CNSs; 18 respondents had been an author on a publication of a regional audit and five had declined to participate in one or more previous regional audits.
Views on the regional audit programme
There was consensus among respondents to many statements relating to regional audit, including the relevance of chosen topics, its benefit to patients and to rheumatology units, its role in promoting discussion within rheumatology departments and its usefulness in trainee education (Table 2). Some statements (marked ‡) did elicit divided opinion, notably regarding how well informed respondents felt about results of regional audits and how effective regional audit is at closing the audit loop. There was consistent disagreement with the statement that ‘regional audit is a “tick box” exercise that does not lead to meaningful outcomes’.
Questionnaire elicited views on regional audit.
Comparison of respondents’ opinions of regional to local audit (Table 3) shows consensus of opinion regarding the greater efficiency and effectiveness of regional over local audit. Out of 68 respondents, 52 (76.5%) agreed that ‘comparing results between units makes audit standards more meaningful’.
Agreement to statements comparing regional audit to local audit.
Analysis of opinions between different groups of healthcare professionals showed no difference between the groups (Table 417). Remarkably similar opinions were held between those respondents directly involved in leading an audit compared to those that collected data or declined to participate in an individual audit.
Amount of agreement to statements about regional audit compared between different groups of healthcare professionals.
Recall of previous regional audits and their outcomes
The majority of respondents employed in the region at the time of a regional audit had heard about each audit. However, fewer people recalled hearing a presentation of the results or reading the peer-reviewed publication (Table 5).
Respondent recall about the different regional audits.
Was change in clinical practice implemented following a regional audit?
Many units reported changes in practice following a regional audit (Table 6). Specific examples include that a regional audit helped in the business case for a local osteoporosis service,9 provided data for negotiations with the local primary care trust to fund eligible patients with biologic therapy,11 re-educated staff about avoiding unnecessary dual energy X-ray absorptiometry scans,12 supported the commencement of annual review clinics13 and reduced empirical prescription of anti-tuberculous therapy in low-risk cases.14
Number of units whose staff reported changes in practice following each regional audit.
The future of regional audit?
There was consensus among respondents that most would like to be involved in choosing topics for future audits and that regional collaboration could be developed beyond audit, for example with research projects or regional guidelines (only three and one respondent respectively disagreed with these statements). There was mixed opinion on whether there should be lay involvement in topic choice.
Discussion
This study has identified many strengths of the innovative programme of regional audits. In particular, there was general agreement that it benefits patients, has the potential to improve hospital services in individual units and offers many advantages compared to local audit including the ability to compare results between units thus increasing the value of the audit. Indeed this comparison between units may also be used to support applications for increased resource provision in order to meet the standards of neighbouring units. Other examples of regional audit have similarly compared, anonymously, performance between units whereas others have not exploited this benefit.18,19 Interestingly, only 27 (40.3%) respondents to this questionnaire thought that, when comparing between units, anonymity was important. In addition, this regional audit scheme was thought to provide specific educational opportunities for SpRs, namely for developing skills in audit methodology, pro forma design and data analysis, essential generic skills that they are expected to acquire during their training.
The degree of agreement with the statements was similar between consultants, SpRs and CNSs suggesting that different healthcare professionals have similar views regarding the regional audit process. Those healthcare professionals who had been less involved in previous audits still held similar levels of agreement to the positive aspects of regional audit, which suggests that the results do not only represent the views of a group of enthusiasts. There was consensus among respondents that regional audit should be a multidisciplinary activity.
Audit ultimately aims to drive changes in clinical practice to improve patient care. A particular strength of this paper is that it attempts to assess the impact of regional audit on local practice. This reveals that significant modifications have indeed been implemented in many units following a regional audit. Regional audit facilitated increased resource provision and supported discussions with local management teams, in addition to changes to clinical pathways. Thus, this regional audit scheme has been a powerful clinical governance tool, with tangible results. Other groups also describe how a regional audit has been used to inform the development of standards of care and sequential national audits have also documented improvements in patient care.5,20 However, compared to national audits, these regional audits have required no external funding. SpRs undertook this work as part of their protected time and the audit department involved took on this project as part of their ongoing funded work.
Areas which invoked divided opinion among respondents included the quality of intra-regional communication. The regional audit programme has been well publicised; however, communication between the steering committee and those collecting data needs to be improved, as does the communication of results. Whether regional audit is effective in closing the audit loop also provoked mixed opinion. The response here may reflect that only one of the regional audits completed so far was a re-audit although Table 6 shows that a regional audit has frequently prompted local re-audit.12 Continuing to publish audits appeared to be more important to some respondents than others which may reflect different stakeholder motivations15; holding this opinion, however, was not related to job level.
Limitations of this study include an acknowledgement that the results may have been skewed slightly by potential regional audit ‘enthusiasts’, even though the analysis did not indicate that previous authors of regional audit publications responded differently to non-authors. Recall bias must be considered in the interpretation of the respondents’ reported involvement in previous audits and description of changes in clinical practice following a regional audit, particularly as the first audit was carried out almost 10 years ago. Also staff will have moved in and out of the region over this time period so not all staff involved in the earlier regional audits were questioned.
Based on the experience and evaluation of this regional audit programme, the following recommendations for setting up a successful regional audit scheme are proposed (Box 2). A regional audit scheme is recommended to other regions and specialties. Specialist societies have been asked to look at methods of developing and assessing clinical quality indicators, an area in which regional audit could play an important role. Future applications include extending the regional collaborative network that regional audit has fostered to include research projects or development of regional guidelines (in areas where national guidelines do not exist).
Recommendations for successful regional audit.
Overall this study has demonstrated that regional audit is widely perceived to be a valuable clinical governance tool improving quality of patient care. This may well prove worthy of future development in light of recent national recommendations for service delivery and the introduction of revalidation.1
Acknowledgements
We would like to thank the WMRSTC for coordinating the regional audit programme, everyone who has contributed to the programme and all respondents to the questionnaire.
- © 2010 Royal College of Physicians
References
Article Tools
Jump to section
Related Articles
Cited By...
- No citing articles found.