Audit has been defined as “systematically looking at the procedures used for diagnosis, care and treatment, examining how associated resources are used and investigating the effect care has on the outcome and quality of life for the patient”.¹ The audit cycle comprises observing practice, setting standards, comparing practice with standards, implementing change and observing new practice. The Royal College of Paediatrics and Child Health recommends that all trainees participate in one audit project every six months.² We made a quantitative assessment of the standard of clinical audit in a children’s hospital over a six year period.

METHODS

One hundred and thirty four audits registered with the central audit office between 1994 and 2000 at a children’s hospital were examined retrospectively. We used the classification advocated by Derry and colleagues.³ The audit cycle was categorised into six stages: stage 1, choosing a topic; stage 2, setting target standards; stage 3, observing practice; stage 4, comparing performance with standards; stage 5, implementing change and planning care and stage 6, re-auditing. The different levels of audit were defined as follows: A “full audit” satisfied five of the six stages of the audit cycle, while a “partial audit” satisfied three of six stages, and a “potential audit” satisfied just two stages. The “planning audit” group included audits where a topic was chosen and only the intentions for audit were outlined. The “no audit” group comprised audits that were not considered to be audit projects. The standard chosen was that 100% of audits should fulfil the criteria for being full audits. According to this method, an audit could be a “full audit” even if it had not been re-audited, provided all the other stages were completed. In addition, we examined registered audits undertaken in the three years from the end of the study period, to ensure that we were not underestimating re-audit activity.

RESULTS

Of 134 audits, 8 (6%) were excluded, as they were research projects. Of the remaining 126 audits, 35 (27.8%) were “full”, 53 (42.1%) were “partial”, 30 (23.8%) were “potential”, and 7 (5.6%) were planning audits (table 1). Of the various specialties, paediatric (34, 27%) and community (20, 15.9%) medicine undertook the most, and the allied disciplines of audiology, nutrition, physiotherapy, and nursing undertook the least (8, 6.3%) audits. Re-audits were undertaken in 28 (22.2%) audits. Recommendations were made in 77 (61.1%) audits.

DISCUSSION

The most striking finding in this study was that the majority of audits carried out in this teaching hospital did not fulfil the criteria for a “full” audit. Only 22.2% closed the loop by re-auditing. Re-auditing after implementing recommendations is essential to ensure that clinical practice has changed and to establish that this improvement is sustained.^1,4 Despite numerous published articles and reviews, the purposes, methodology, and potential of audit are unclear to many physicians.⁵ Failure to complete all stages affects the outcome and defeats the very purpose of audit.

The reasons for failing to perform the full audit are many. Often, the responsibilities for performing the audits fall on junior medical staff and the rotational nature of their posts hinder completion of the audit cycle.⁶ Data collection is often by retrospective analysis of case notes, which can be time consuming, inaccurate, and labour intensive.⁴ Specific training on audit methodology should be part of early departmental teaching for junior doctors. Some junior medical staff undertake audit projects on their own unknown to the central audit office and this can seriously affect the quality of audits. The supervising consultant should be the link in re-auditing by designating re-audit projects when junior medical staff start new posts, as well as deciding a suitable time frame for re-audit. Some changes can be achieved rapidly while others take longer; a re-audit after a longer interval may evaluate more effectively the sustainability of a change in practice. This is an important factor in determining the frequency of auditing a department’s audits and is best left to the supervising consultant. There are even practical difficulties in getting an audit project ready for presentation before the junior doctor leaves the department. Different specialties should jointly undertake audits, as there is often a significant degree of overlap in the multidisciplinary care of a patient. Only a minority of audits (8/126) involved the allied disciplines of audiology, nutrition, physiotherapy, and nursing (table 1), questioning how multiprofessional the majority of audits were. We recommend that all audit departments undertake an audit of their audit projects annually to monitor completion of the audit cycle and that the central audit department should track the progress of audits and send reminders when a re-audit is due. This would promote awareness and lead to better quality audits. We propose to examine the impact of this initial quantitative assessment of audits on qualitative change achieved in clinical practice, modifying current local audit training, and implementing local departmental mechanisms for steering audit.

In conclusion, unless audits are well planned and properly undertaken, they may end up as an unproductive activity done by junior medical staff in an attempt to fulfil the educational requirements of the post. Re-education on the basic principles of clinical audit and regularly “auditing the audits” will help derive maximum benefit from good quality audits.