Competency mapping in quality management of foundation training
Abstract
Competency-based curricula focus on outcomes in terms of application of knowledge and acquisition of competencies. The aim of this exercise was to analyse posts and programmes for potential training outcomes. A mapping process against the UK foundation curriculum was designed.1 Data were gathered from foundation directors using red/amber/green judgements for each post. After submission no correspondence was needed on any programme about educational arrangements, suggesting the process had a significant formative effect as well as assessing outcomes. Certain competencies were problematic across the majority of host providers, allowing programme directors to ensure deficiencies are compensated by planned teaching or simulation training. Widespread difficulty in achieving these competencies raises the question of whether they should be included within the national curricula. Development of competency-based training is a complex, multistep process. However, it is possible to analyse it in a large programme of trainees in the setting of the modern, busy NHS.
Introduction
Postgraduate medical training in the UK has undergone fundamental changes in the last decade. Modernising Medical Careers (MMC) started with foundation training (FT), which has been fully operational since August 2005.
The curriculum for FT is broad and competency based,1 the outcome of the programme being doctors with defined abilities rather than trainees who have had defined experience. Traditionally, postgraduate medical training has been gained through practise within a programme of pre-determined fixed length. FT aims to offer experience in primary care and also provides opportunities for experience in smaller specialties and academic medicine.2
Much of the literature regarding competency-based training (CBT) originates from the USA,3,4 and similar frameworks have been drawn up in Canada, Australia and the UK, among others.5 In the UK, a move to competency-based education has been encompassed within MMC, seen firstly with the implementation of FT. Similar competency frameworks are the basis for specialty-specific curricula developed by the royal colleges.
Measuring competence is undoubtedly more straightforward in specialties such as surgery with well-defined procedures and processes in which competency can be judged. It has been recognised that traditional evaluation methods, through knowledge examinations or limited skills testing, are inadequate to measure competency in all domains.6 A ‘toolbox’ of methods has been suggested, which can demonstrate competence in each core clinical area using triangulation of a number of tools.7 There have been efforts to measure behavioural objectives which can be expected of trainees at varying levels of seniority.8
Changing the focus from ‘fixed-length variable-outcome’ programmes to ‘fixed-outcome variable length’ (CBT) programmes, might possibly reduce the time required for training,9 and with it cost, but this would have huge implications for workforce planning. Constrained by the necessity of service provision, it is difficult to see how a fully flexible CBT system could be developed. The recent introduction of a 48-hour week with the European Working Time Directive (EWTD) has revived arguments over the time required to gain adequate experience, particularly in procedural-based specialties.
Methods
The North Western Foundation School covers a wide geographical area from Barrow-in-Furness to Stockport, with over 1,100 trainees, 16 programmes, 19 sites and 12 employers. Each trainee rotates through six posts over a two-year period and is expected to achieve all the generic competencies as set out in the UK foundation programme (FP) curriculum.1
Usually 12 months before the programme is due to begin, the employing trusts have been asked to submit a spreadsheet with all training tracks outlined in terms of specialties and with some information about out-of-hours banding payments, with free-text explanation as to how posts fulfil the aims and objectives of FT. With little structure to it, this section has always been somewhat subjective as regards attainment of competencies. The body judging the type of experience on offer in a particular post would be the host specialty. Competencies would not directly be assessed and were considered more by inference from the specialty the trainee was placed in. The great majority of submissions required correspondence from the school director for clarification of the intended rotations.
The aim of the exercise outlined in this paper was to allow a more structured analysis of the individual posts and therefore the two-year tracks in terms of training outcomes ie the competencies outlined in the foundation curriculum. The mapping process was designed during training courses for foundation teams in the school. Forms for the mapping of competencies from the foundation curriculum for each post and for summarising six posts across a track were designed. It was agreed that these documents would be submitted as part of the normal approval process outlined above. It was suggested that data be gathered from trainers for each post in terms of the FT competencies using a red/amber/green judgement for each, where red implies no opportunity to gain competency, amber provides limited opportunities, and green is ample opportunity.
Findings
The extra complexity and amount of work was found to be challenging by some teams. Of the 16 programmes in the school, six submitted paperwork on time. Unprecedented second and third meetings had to be convened for the eight programmes who submitted a week late and the final two who were a fortnight late. All programmes did, however, eventually submit paperwork.
Of the 186 tracks in the region, there were 29 non-returns from individual posts (mainly academic placements) which left gaps in a track's matrix, but this did not preclude assessment of the competencies delivered in practice. Figure 1 shows examples of tracks which are predominantly favourable for most competencies and should be no cause for concern. Some tracks were submitted which had less favourable combinations of posts as in Fig 2.
Overall all the tracks provided opportunities to achieve set foundation year competencies. Very few of the competencies were shown as red in all six posts in a track. Where competencies were not shown as green in any of the six posts in a track, these were mainly procedural in nature. Central venous line insertion, joint aspiration and lumbar puncture were particularly noted, with 48% of tracks in the North West providing only limited or no opportunity for central line insertion. Of the generic competencies, nutrition, complaints handling, and initiation of ALS were shown to be problematic. Table 1 shows the percentage of tracks in which opportunities to achieve competencies were not shown as green in foundation years 1 and 2 and through the track overall. It was seen that all the ‘problematic’ competencies (ie those not shown as green in any of the six posts) were judged as green in the teaching programme.
Discussion
The competency mapping exercise has provided direction for the foundation school and its programme directors. It has highlighted areas of deficiency within clinical posts which can be compensated for by targeted training, extra attachments, simulation or other planned teaching.
One unexpected feature of the exercise was that after the submission meetings no correspondence was needed by any programme about clarification of the rotations or suggested re-arrangements. It would seem that the process had a significant formative effect on the design of the programme development as well as being an assessment of the output of the posts.
This assessment of our FP tracks has, for the most part, shown that the trainees have adequate opportunities to achieve the deemed competencies by the end of the programme. Only 18 essential competencies did not have ample opportunity for achievement in all tracks by the end of FT. However the FP is designed as a spiral model whereby a trainee should achieve all competencies by the end of the F1 year, and consolidate these by the end of F2. It can be seen that the numbers of tracks without a green post in F1 or in F2 individually are significantly higher than those without a green post anywhere in the track as a whole. When assessed separately only 14 competencies were achievable in both years in all tracks. This raises concerns about the achievability and practical implementation of this spiral model. Mapping of competencies would be especially important if FT is attempted in two disconnected year-long posts or in ‘themed’ tracks containing a preponderance of certain types of experience.
Areas were highlighted which were not being reliably covered on all programmes over the two years, including nutrition, complaints and some procedures although by the nature of the exercise there was no further detail available on the reasons for these limitations. It can be seen that some competencies could only be achieved in certain posts, such as joint aspirations in a rheumatology placement. If this is the case, it could be argued that all trainees should rotate through such posts. An alternative discussion would be whether such competencies are achievable and appropriate at all. While competence in some of the procedural skills is not deemed mandatory within the FP curriculum, the question remains as to whether the competencies should be included even for illustrative purposes if there are such difficulties in achieving them. Should central line insertion, for example, be suggested as a competency from all trainees or is it unrealistic at this stage of training? If not, then the curriculum needs to reflect this at a national level.
Interestingly the competencies less likely to be achievable in F2 related to acute medical management such as taking part in advanced life support and resuscitation decisions. This is probably a reflection of the nature of the posts done in F2 which, in the North West, all include a placement in general practice and tend to be more specialty based, for example public health or paediatrics. It would be expected, therefore, that these posts would offer less experience and opportunities in these areas. Although this offers the students a wide and varied experience it could be concluded that they may become deskilled in essential areas in their F2 year.
As seen in Fig 2, one post was found to have cause for concern in meeting the majority of FT procedural competencies. It could be inferred from this, unfairly or not, that this post has little value within the training programme. By this method, however, no account has been taken of its educational value overall. It could be possible that programmes or departments may over-emphasise the educational value of some posts and be optimistic in their judgement of the competencies achievable. This is a limitation of the exercise and further work to assess trainee views in conjunction with those of trainers is currently being undertaken. It will be interesting to assess concordance between those providing and those receiving FT.
Conclusion
Development and organisation of competency-based training is a complex, multistep process. However, it is possible to analyse objectively in a large programme of trainees in the setting of the modern, busy NHS.
Competency-based training requires regular assessment of the experiences provided for competency attainment. Such a process as described in this article highlights problem areas of the curriculum and is also formative for education providers. It should be remembered that this was solely the perspective of the trainers and future studies will triangulate this work with the actual experiences of the trainees.
Nationally a similar process could be employed for assessment of programmes. It would be extremely valuable to identify whether the same problem competencies are duplicated across the UK, and may even lead to re-evaluation of the curriculum content. It is envisaged that such quality assurance work could be extended to other programmes such as core medical training and higher specialty training.
Acknowledgements
The process described in this paper developed from sterling work led by Fran Shinks, medical education manager, Bolton, and Dave Highley, foundation programme director, Lancaster. Thanks also to James Fishwick, foundation school manager.
- Royal College of Physicians
References
- ↵
- Department of Health
- ↵
- Modernising Medical Careers
- ↵The Accreditation Council for Graduate Medical Education outcome project: an introduction, www.acgme.org/outcome/project/proHome.asp.
- ↵
- ↵The Royal College of Physicians and Surgeons of Canada, The CanMEDS Physician Competency Framework, http://rcpsc.medical.org/canmeds/index.php.
- ↵
- ↵A product of the joint initiative of the ACGME Outcome Project of the Accreditation Council for Graduate Medical Education (ACGME), and the American Board of Medical Specialties (ABMS), 2000 www.acgme.org/Outcome/assess/Toolbox.pdf.
- ↵
- Carraccio C,
- Englander R,
- Wolfsthal S,
- Martin C,
- Ferentz K
- ↵
Article Tools
Citation Manager Formats
Jump to section
Related Articles
- No related articles found.
Cited By...
- No citing articles found.