Royal College of Physicians medical record keeping standards audit

Editor – In an environment where thousands of clinical audits are completed each year you would be forgiven for assuming that the maths behind the audits would be clear cut. However, in reality it is questionable how many audits contain subtle inconsistencies in the analysis of results that can dramatically affect the overall outcome of the audit. These oversights may not be picked up on first glance, if ever.

I became aware of the complicated nature of statistics in relation to audits while undertaking the ‘Royal College of Physicians (RCP) medical record keeping standards audit’ using the provided audit tool. The tool measures a department's performance against each of the RCP set standards by averaging the percentage scored for each standard in each set of medical records. This gives an average percentage performance for the sets of records. While the technique of averaging the percentages is not mathematically incorrect it is questionable whether this method is the most appropriate for this set of data as it assumes that all the entries have identical weighting. An example of this is that if one set of records with 99 pages scored 99/99 or 100% and another set with one page scored 0/1 or 0%, the average of these would be 50%. It may be more appropriate to consider a department's performance across all pages in all sets of records. In this case, the overall score would have been 99 out of 100 pages, or 99%. With such a large difference between the outcome of these methods it is important to understand the calculations before making any change to practice based on the results of this audit.

Here you can see that a simple and seemingly minor variation in the method of results analysis can produce a considerably different set of results. When conducting an audit using a pre-configured audit tool, you will likely take it for granted that the tool is making the calculations that you would expect. It is important to understand what the tool is trying to achieve and scrutinise the underlying statistical methods used to analyse the results. With so many audits being completed it is impossible to say how many inadvertent errors in the interpretation of results have gone unnoticed, although it would suffice to say that this is not a one-off.

• © 2010 Royal College of Physicians