Specialty certificate examination in gastroenterology
The aim of training in medicine is to create future consultants with the knowledge, skills and attributes required to provide high quality care to patients. Several recent pressures have resulted in changes to medical training. As a consequence, assessment has become more important with the need to test both the knowledge base and workplace skills to ensure that trainees are ready to face the complexity of demands they will meet as a new consultant.
The medical royal colleges and specialist societies have taken a lead in providing guidance on how to improve training and assessment. The need to ensure that trainees have the knowledge to practise safely is vital. MRCP(UK) has provided a successful platform for launching trainees who have a good fundamental core of knowledge and skills in general medicine that allows them thereafter to diversify into various medical specialties. The MRCP(UK) skill base has now led to the development of specialty certificate examinations (SCEs) in a range of medical specialties (www.mrcpuk.org/SCE/Pages/Home.aspx). MRCP(UK) has over many years developed assessments that reflect the range of experience required of trainees (validity), are reproducible (reliability) and are set at the required standard.
Gastroenterology is a rapidly developing medical specialty with a large body of knowledge, much of which is recent, and many and varied sources of national and international guidelines giving advice on ideal management. The training curriculum has recently been completely revised after significant consultation. It is also a very demanding service specialty with large outpatient clinics, busy endoscopy services, multidisciplinary teams, subspecialty development and for most a continued commitment to acute general medical receiving. Daily work requires good communication skills, diagnostic acumen, manual dexterity, ability to assess complex images and the need to work as part of a team. Assessment has to reflect this complex interaction of skills. Gastroenterology is a visual specialty that requires the regular interpretation of endoscopic, radiologic and histological images in relation to the individual patient's problem.
Colleges have followed best educational advice as to how to assess such complex clinical skills. MRCP(UK) has developed the ‘best of five’ format question to test real clinical scenarios and offer five feasible alternative answers to a diagnostic, investigation or management problem. One option is always agreed to be the most correct. This format reflects the clinical dilemmas and options available in practice. The full curriculum is tested by following a blueprint that ensures that a fixed proportion of all subjects are selected for each diet of the examination. The reliability of the examination is ensured by providing sufficient questions (200) to make certain that the assessment would give reproducible results in each cohort of candidates. The standard for each diet is set in advance by a standard setting group who consider the complexity of each individual question and award it a mark, which then contributes to an overall criterion referenced pass mark for that diet.
The SCE in gastroenterology has therefore been developed to allow fair and accurate assessment of trainees' abilities to assess and respond to clinical problems. It is currently held once per annum and is computer based with two three-hour papers containing 100 questions each. The computer-based testing allows many venues to hold the SCE simultaneously throughout the UK and internationally. The SCE is designed for UK specialist trainees but overseas candidates who hold MRCP(UK) and have relevant training can also apply. UK trainees who commence training from August 2007 onwards will be required to pass the SCE in order to be awarded a Certificate of Completion of Training (CCT). They may use the post-nominal MRCP(UK) (Gastroenterology) after award of the CCT. All successful candidates will be awarded a certificate in gastroenterology. The SCE, like workplace assessment, will therefore become an essential passport to completion of training. Trainees are recommended to attempt to pass the SCE by the time of their penultimate year of assessment and therefore should ideally take the examination during years ST5 and ST6 when they will have gained sufficient experience. The SCE does require extensive knowledge of gastroenterology and the curriculum ranges from basic physiology to complex clinical scenarios. The standard expected of trainees is the knowledge and skills that should have been acquired after three or four years of specialty training but additional reading of textbooks, guidelines and journals is required to ensure that they have an awareness of up-to-date knowledge and less-common conditions. The expectation is that all UK trainees should be able to pass the examination readily at first or second sitting. Images are regularly shown throughout the examination as they constitute so much of the daily practice of a clinical gastroenterologist.
Preparation for the SCE has been underway since 2007 with the appointment of consultant gastroenterology members of the British Society of Gastroenterology (BSG) to the question writing group (25 members), examination board (10 members) and the standard setting group (eight members). All question writers were trained at the outset and have now attended five two-day meetings to edit and accept questions written by the group. This has resulted in a bank of some 1,200 accepted questions. Questions are then selected by the examination board using the agreed blueprint to provide 200 approved questions for each diet. The standard setting group then establishes a pass mark for each diet which reflects the complexity of that set of questions and also further reviews any ambiguities that questions may present. College editors establish a standard format and house style for questions to ensure consistency. Members of the examination board then proofread the SCE and run through the computer presentation to ensure that formatting is correct and that answers are coded correctly before the diet is held. Images are also viewed on screen to ensure that reproduction detail is adequate and that orientation is correct.
As the SCE is computer based, the marking is all electronic and the assessment of outcome is rapid. However, the examination board takes great care to assess answers to each question to see if any anomalous patterns of answering suggest an ambiguity or incorrect coding. Performance and reliability of questions are examined and documented for subsequent use. If any question has not performed as expected, it can be removed from the final assessment before an amended pass mark is chosen. Results are then released to candidates within a short period and pass rates and performance of individual questions analysed by the board.
Gastroenterology was fortunate in having a team of experienced MRCP(UK) examiners within the specialty and strong support from both the BSG and the Trainees in Gastroenterology group, all of which enabled an early start in the process. The very first SCE was held in gastroenterology in June 2008. Two diets have now been held. The first diet had a small number of candidates but at the second, in November 2009, 105 candidates sat with a pass mark of 64.9% and an overall pass rate of 61%. The cohort at this diet were not representative of the UK trainee population but the future expectation is that UK numbered trainees will have an 80–90% pass rate once they have sufficient experience. This will ensure that the SCE is effective in encouraging trainees to prepare and acquire the skills required to become a consultant gastroenterologist but that it will also not be an impediment in their progress through this stage of training.
The process of planning, developing and standardising the SCE has been a substantial learning curve for members of the examination groups. However, it has been an enjoyable and very informative challenge from which I believe all of us have learned. I very much appreciate the enthusiasm and support that I have received from my colleagues and the BSG in enabling us to create the SCE in a very thorough and robust form, in a relatively short period of time. It will undoubtedly develop considerably over the years but is clearly now well established as part of an important package of improvements in UK medical specialty training. Trainees may initially baulk at the thought of the cost or time required to prepare for the SCE but I strongly believe that they will be better qualified for modern practice requirements as a consequence.
- © 2010 Royal College of Physicians
Article Tools
Citation Manager Formats
Jump to section
Related Articles
- No related articles found.
Cited By...
- No citing articles found.