Improving international medical graduates' understanding of the UK appraisal system: an interventional study
ABSTRACT
International medical graduates (IMGs) face countless challenges when migrating to a new healthcare system, of which understanding the UK appraisal and revalidation system is one. We investigated whether provision of educational material on appraisals would improve IMGs' confidence in their understanding of the UK appraisal system. A prospective pre-post interventional study aimed at IMGs was carried out between 25 February 2022 and 9 March 2022. A mixed-methods survey was undertaken pre (n=519) and post (n=63) intervention. The pre-interventional survey highlighted IMGs' lack of experience and knowledge of the General Medical Council (GMC) appraisal and revalidation process. Postinterventional responses showed a significantly higher confidence rating in IMGs' understanding of the UK appraisal process (p<0.001). Utilising simple educational tools can be beneficial for IMGs to gain confidence in navigating appraisals and help bridge the attainment gap when entering a new healthcare system. Barriers, such as lack of knowledge, can be easily rectified without the need for significant investment.
Introduction
Appraisals, or performance reviews, are regularly conducted in healthcare organisations as a means for the appraisee to reflect on practice and develop agreed targets. The Standing Committee on Postgraduate Medical Education (SCOPME) outlines appraisals as a two-way process with the purpose of identifying the educational and developmental needs of an individual to help them progress with agreed targets.1 In the UK, the General Medical Council (GMC) act as the regulatory body for doctors. Doctors on the medical register in the UK must undertake the revalidation process every 5 years to demonstrate to the GMC that they are up to date clinically and fit to practise. It is considered a key element in improving and maintaining patient safety and the quality of care delivered.2 The GMC describes revalidation as ‘a continuous process that all doctors must participate in, in order to retain their licence to practice medicine in the UK’.3 Annual appraisals have been mandatory for all doctors working in the UK since the start of revalidation by the GMC in December 2012.2 During their appraisal, doctors must reflect and demonstrate their practise in accordance with the GMC guidance Good Medical Practice to a trainer appraiser.4,5
A GMC report looking at revalidation data from the initiation in 2012 to 20186 highlighted that doctors who received their Primary Medical Qualification (PMQ) from outside the UK, including within the European Economic Area (EEA) and international medical graduates (IMGs), were less likely to revalidate their license compared with UK graduate colleagues. Only 71% of graduates from the EEA and 74% of IMGs had revalidated over the years, compared with 78% of UK graduates. When considering differences in outcomes by ethnicity, the report found that those from ethnic minority backgrounds were less likely to revalidate (73%) compared with their counterparts from a non-ethnic minority background (78%). The data clearly present a disparity in revalidation numbers between native and non-native PMQs and again between ethnic minority and non-ethnic minority doctors. A recent GMC Workforce Report 2022 highlighted that the number of IMGs in the UK medical workforce has increased by 40% over the past 5 years,7 In 2021, more IMGs joined the UK workforce (50%) than UK graduates (39%).7
IMGs often migrate and enter the UK medical system in search of greater training and to enhance their professional prospects. During this migration, IMGs confront countless challenges, including separating from their established support networks, adapting to different cultural settings, familiarising with foreign terminology and facing a new medicolegal system. Therefore, IMGs require proper inductions into the system to help them practice within the UK medical workforce without complications. Given the statistics that show that those with a PMQ from outside the UK are less likely to revalidate,6 and combining this with data that show an increasing number of IMGs entering the UK workforce,7 there is a greater need than ever to ensure appropriate support and guidance for IMGs around appraisals and revalidation. With increasing pressures on the UK healthcare system, retention of doctors and supporting them through the appraisal and revalidation process are vital for ensuring that the common goal of patient safety is reached.
A qualitative survey evaluating the challenges faced by IMGs raised a multitude of issues.8 IMGs were found to have difficulty understanding the portfolio system, with many struggling to grasp the meaning of appraisals and revalidation. This was further compounded by supervisors who lacked a logistical understanding in applying the portfolio system to doctors not in training. The lack of information before registration, the inconsistent provision of training and support, and the isolation from proper organisation during non-training posts were the most common difficulties experienced by IMGs, according to a further qualitative study.9 All these factors have key roles in the development of a competent portfolio that is required to progress through the appraisal and revalidation process. A realist synthesis,10 conducted in 2016, found that those organisations that had successfully integrated IMGs into new healthcare systems provided unique programmes that used a compassionate learning environment to accommodate the distinct needs of IMGs while simultaneously offering support from peers and supervisors throughout.
We used the Knowledge to Action Framework (KTA)11 to identify and understand the barriers that IMGs face with the appraisal process and to help develop and implement a simple intervention of tailored education. The seven-step framework, which was first developed in Canada in 2006, provides the user with a structured approach to make change. To identify the initial gap between knowledge and practice, we proposed a mixed-methods survey, comprising quantitative and qualitative metrics targeting IMGs. We then proposed an educational webinar as our intervention to tackle lack of knowledge of appraisals. We proposed a further mixed-method survey to understand knowledge attainment post webinar. Using the final stages of the framework, we planned to evaluate our intervention.
The study described here was undertaken to determine whether provision of educational material on appraisals would improve IMGs' confidence in their understanding of the UK appraisal system. Our null hypothesis was that the medians and distributions of confidence ratings would be the same pre- and post-educational material.
Methods
The Specialty Doctors and Associate Specialists (SAS) and Locally Employed Doctors (LED) Forum was established in 2020 with the aim to provide SAS and LED with recognition, support and mentorship to progress in their careers. The Forum members comprise a mix of IMGs and UK graduates. With a large proportion of LEDs having migrated from abroad, regular attendees of webinars are IMGs. The Forum regularly offers educational material online and free of charge to encourage all doctors to have access to the same knowledge.
We carried out a prospective pre–post-interventional study aimed at improving IMGs' knowledge of appraisals at all levels. Using the KTA framework,11 we designed our intervention. To identify the knowledge–practice gap, an initial mixed-method survey targeted at IMGs was distributed electronically among the social media platforms of Facebook, Twitter and Instagram between 25 February 2022 and 6 March 2022. The aim was to gain insight into IMGs' understandings of appraisals (n=519). The mixed-methods survey (five-point, quantitative questions and open-text, qualitative questions; Box 1) assessed the IMGs' current knowledge and confidence as well as understanding the barriers to portfolios and appraisals.
The responses from the initial survey were subsequently utilised to aid the development of an educational webinar to tackle areas of improvement. A focus group meeting with Forum members was held to select topics to be covered in the educational material and decide how knowledge would be shared. A 90-min educational webinar was carried out on 9 March 2022. The educational webinar covered: (1) introduction to portfolios and types of portfolio; (2) GMC appraisal and revalidation processes; (3) contents of the portfolio; and (4) a Q&A session. Those who completed the initial survey were encouraged to attend the educational webinar (n=106).
The attendees of the webinar were then invited to carry out a follow-up mixed-methods survey electronically immediately after the webinar. The purpose of the survey was to evaluate the outcomes of the intervention on IMGs (n=63) by again assessing their knowledge and confidence post intervention (Box 2).
The non-parametric Wilcoxon–Mann Whitney and Median tests were used to statistically compare five-point confidence ratings, on the GMC appraisal and revalidation process, pre and post intervention. Statistical tests were analysed using IBM SPSS Statistics software Version 28. p<0.05 was considered statistically significant
Results
A total of 519 IMGs completed the initial electronic survey, of whom 52% (269/519) were not working in the UK healthcare system. The remaining 48% (250/519) of participants were working in the UK healthcare system and distributed at Foundation (15%), core trainee (24%) and registrar (9%) levels in both training and non-training posts.
We asked all participants to give an insight into their experience of using portfolio systems (Table 1). Of the IMGs who were not working in the UK healthcare system, none had a portfolio that they used regularly. Only 5% of the IMGs who were already working in the UK had a portfolio they used regularly.
When asked about their experience of appraisals, 90% of the IMGs who were not working in the UK and 78% of those working in the UK strongly agreed or agreed that they had never undertaken an appraisal before. Only 4% of the IMGs who were not working in the UK and 8% of those working in the UK felt confident in their knowledge of the GMC appraisal and revalidation process (Fig 1).
IMGs were questioned about areas of the system that they felt they needed more knowledge on. Common keywords from qualitative questioning included ‘portfolios’ (37%) and ‘appraisals’ (27%). Examples of areas the IMGs needed more knowledge on included:
‘Information about appraisals and e-portfolio especially for the IMGs. When to start working on e-portfolio and how to maintain it. When do we require appraisals etc? What is different while working in UK (e-portfolios do exist in middle east and back home in Asian countries but what is different about having one in NHS?’
‘How do I use the E-portfolio to suit future career aspirations as well as present job requirements?’
‘E-portfolio maintenance and precise details of each section. Mini-CEX, CBD etc with examples’
A total of 63 IMGs completed the post-educational webinar electronic survey. Figure 2 shows a breakdown of responses to the quantitative questions in the post-intervention survey. All the survey participants were working in the UK. After the webinar, 100% of the attendees agreed or strongly agreed that they could clearly identify and clarify what a portfolio was and the different types of portfolio. In addition, 95% agreed or strongly agreed they could differentiate between the different contents of a portfolio; 97% agreed or strongly agreed that they could define and differentiate the GMC appraisal and revalidation system; and 94% of the attendees felt more confident in their knowledge of the GMC appraisal and revalidation process after the webinar. Figure 1 compares pre- and post-intervention confidence ratings of IMGs working in the UK.
Statistical analyses showed a significant difference (p<0.001) in response and, therefore, we rejected the null hypothesis that the medians of confidences and the distribution of confidences would be the same pre and post intervention.
Discussion
In our experience, IMGs lack familiarity with portfolios or appraisals before joining the UK healthcare system and, therefore, need support with overcoming this barrier to successfully work as part of the system. Our study demonstrates that providing a simple solution, such as educational resources in an interactive format, increases confidence significantly in IMGs. The study used an interventional design that could easily be replicated and support numerous people simultaneously. With 97% of members agreeing that they could now easily define and differentiate the GMC appraisal and revalidation system after a relatively short 90-min online webinar, the investment needed to support IMGs in the process need not be high.
IMGs face a multitude of challenges and barriers when entering a new healthcare system. In accordance with the responses received during a qualitative study,9 our experience found that a significant number of IMGs lacked confidence in, and knowledge of, appraisals. The variety in qualitative responses to our initial survey revealed that, in general, IMGs need more educational provision around the whole appraisal process. With increasing numbers of IMGs entering the UK healthcare system,7 more materials and resources need to focus on supporting these doctors on arrival and our study showed these resources do not need to be expensive or time consuming. Data shows us that ethnic minority doctors and IMGs are negatively impacted by revalidation6; thus, more needs to be done to bridge this inequality gap. Noaman et al. described the attainment gap between IMGs and UK graduates when it came to applying for a national training number (NTN).12 They found that portfolio was the lowest scoring section for IMGs applying for an NTN. The underlying cause for this finding was not explained, although a poor understanding of the portfolio system could be a potential factor, as demonstrated by our study. Woolf et al. highlighted the lack of knowledge about UK systems as a cause of differential attainment, and our study demonstrated that providing education can be a powerful tool that is potentially life and career changing.13 The piloting of standardised voluntary induction programmes for IMGs was recently introduced to help tackle some knowledge barriers.14 Healthcare organisations, such as hospital Trusts, should also focus on providing specific educational materials during induction and ensure that these are easily accessible to doctors to refer to when required.
We recognise the limitations of our study, which might have some impact on the validity of our findings. Our surveys did not question how long the doctors had been working while abroad or within the UK healthcare system. Therefore, we did not consider additional factors that might impact an IMG's knowledge of the UK medical regulatory system. For example, IMGs who have worked for numerous years are more likely to have knowledge and experience of appraisals compared with a newly qualified IMG. Our pre- and post-intervention surveys were distributed 12 days apart to allow the Forum adequate time to develop the appropriately tailored intervention. We acknowledge that, within that period, other factors might have had a role in contributing to respondents' knowledge of appraisals. We tried to limit any further time gap by asking respondents to complete the post-intervention survey immediately after the webinar. We recognise that this was a single-study intervention and, therefore, has limitations in informing policy decisions. We look to repeat this study on a greater scale to confirm the reliability of our findings.
Conclusion
Our interventional study demonstrated that utilising simple educational tools, such as an online webinar, can be beneficial for IMGs to gain confidence in navigating appraisals when entering a new healthcare system. We understand that IMGs face challenges with the revalidation process and have little experience and/or knowledge of using portfolios or undertaking appraisals. This study shows that barriers, such as lack of knowledge, can be easily rectified without significant investment. Organisations could support IMGs by introducing educational material, such as online videos or easily accessible guides, which are sustainable options and can replace a one-off webinar to be utilised on a larger scale. Our findings show the attainment gap between IMGs and UK graduates might be preventable with proper education provision. Other factors, such as improper supervision and inconsistent provision of training, remain important barriers to IMGs' career progression and must also be tackled further.
Acknowledgements
We thank the members of the SAS & LED Forum who dedicated their time to not only making this study possible, but also for endlessly supporting international medical graduates in their unique journeys.
- © Royal College of Physicians 2023. All rights reserved.
References
- ↵
- Standing Committee on Postgraduate Medical and Dental Education
- ↵
- NHS Revalidation Support Team
- ↵
- General Medical Council
- ↵
- NHS England
- ↵
- General Medical Council
- ↵
- General Medical Council
- ↵
- General Medical Council
- ↵
- ↵
- ↵
- ↵
- ↵
- Noaman I
- ↵
- Woolf K
- ↵
- Hodkinson J
Article Tools
Citation Manager Formats
Jump to section
Related Articles
- No related articles found.
Cited By...
- No citing articles found.