Scholarship, teaching practice and educational responsibility: Issues in designing and implementing a quality improvement and evidence-based practice module in the undergraduate curriculum
ABSTRACT
As part of a review of the undergraduate medical curriculum at King's College London, a module preparing students to undertake a quality improvement project (QIP) was developed.
Using an illuminative evaluation method, the successes and challenges of the module were identified.
The student experience lay along a continuum. At one end, QIPs enabled some significant improvements within trusts and primary care. Projects were presented in their clinical settings and at national and international conferences, and were published. At the other end of the continuum, students struggled to find an actionable project or have early and regular communication with their supervisors.
Poor implementation of the module created challenges. These included misunderstanding of module requirements by students and supervisors, lack of clarity about what a feasible undergraduate project comprised and logistical problems when students moved from their QIP site to their next rotation. Travel back to the QIP site to complete projects involved missing scheduled teaching in their current rotation. Supervisors were unsure how to assess group projects.
Key successes included students feeling better prepared to undertake QIPs, students developing a better understanding of the dynamics of clinical settings and teams, and how to manage these to progress projects.
Introduction
In this paper, we aim to share key lessons about designing and implementing a quality improvement (QI) module and open debate about the relationship between scholarship, curriculum development and the experiences of students and faculty.
As part of a radical review of the undergraduate medical curriculum at King's College London (KCL), a new module was introduced to prepare students to undertake a quality improvement project (QIP) in 2016. QIPs were one of a number of new curriculum elements providing students with contemporary preparation for their roles as NHS doctors. The quality improvement and evidence-based practice (QIEP) module focused on the role of a doctor to actively improve care processes. This acknowledged the role of the doctor not only as a provider of medical services but also as an agent for change for improvement. The module design was a collaboration with the Institute of Healthcare Improvement (IHI) and draws upon the IHI Open School Quality Improvement Practicum.1
Taken in their penultimate year at KCL, the module engages students in a QIP over a 6-month period. The aim is for students to develop the knowledge, understanding and skills necessary to engage in QI methodology, and develop team-working skills with both their peer group and a clinical team. At least four ‘face-to-face’ tutorials are specified between QI supervisors and students, and other teaching is provided online. Protected module time is 3–4 hours a week on a specified day.
A limited pilot was undertaken that indicated both the potential of the module and some challenges. Such was the optimism of module designers and education leaders that greater emphasis was placed on the potential over the challenges. The module was implemented in both hospital and general practice clinical contexts for a cohort of 400 students.
We share some of the lessons identified within an educational study of the first 2 years of implementing the module, using a social science methodology called ‘illuminative evaluation’.2 This methodology has been used in education, medical education, healthcare and other disciplines.3
Methods
We developed four overarching research questions.
In its current form, is the QI module sustainable?
In the hospital and general practice clinical settings, what are key enabling and inhibiting factors when engaging in QIP?
How did the students and teachers experience and value engaging in QIPs; with what effects?
How was the curriculum interpreted and understood by those who taught and engaged with it?
These research questions required a methodology that is interpretivist, focusing on the complexity of contexts, the processes of learning and teaching, and the extent to which aims are realised. This provided a rationale for choosing ‘illuminative evaluation’.
Research methods included semi-structured focus groups and interviews to capture the student experience and interviews to capture the experience of faculty and clinical supervisors. These methods enabled the pre-defined questions to be explored while opening a space for participants to raise issues that were important to them and identify any unintended effects or concerns. Focus groups and interviews lasted approximately 1 hour and were digitally recorded. Over the first 2 years, five focus groups and 20 interviews were held, involving a total of 70 students. Data from focus groups was thematically analysed to identify key themes. Four members of the research team were allocated some focus groups and interviews. They independently undertook the analysis of their allocated data. Each brought their analysis to a group meeting during which the themes were reviewed in relation to the supporting data and accepted or refined through group discussion.
A scoping review was conducted over a 4-month period from April 2017 to June 2017; databases searched include Embase, Ovid MEDLINE, Web of Science, Cumulative Index of Nursing and Allied Health Literature, and the Cochrane Database of Systematic Reviews.
Broad themes used in the literature search included ‘quality improvement’ and ‘medical school’ or ‘undergraduate’, combined using the Boolean operator ‘and’. A combination of Medical Subject Headings (MeSH) terms and free text were used to maximise the search. Additional articles were identified from hand searching reference lists. Duplicates were removed. Articles were included if they examined educational interventions for undergraduate medical students specifically. Those that included educational interventions for postgraduates were excluded. The intervention must have been described in detail and enabling factors/barriers to successful curriculum implementation addressed. Expert opinion articles were also included as these provided insights into the wider national challenges affecting QI curriculum implementation, with a particular focus on factors affecting faculty members.
Titles were initially screened and any that were obviously of no relevance were removed. Where there was uncertainty about inclusion, an opinion was sought from an expert in medical education. All final included papers were reviewed by two medical educationalists and an academic medical educational foundation year 2 doctor. Themes were tabulated, reviewed and discussed by the research group.
Resources to undertake the evaluation were limited in terms of personnel. Data were collected by the principal investigator lead with newly graduated doctors in their second year who had been selected to undertake an academic medical education rotation. During this 4-month rotation the principal investigator and junior doctors collected and analysed data.
Results
Students and faculty proved difficult to recruit due to student workload and clinical commitments, respectively.
In the first year, the evaluation team ran focus groups and interviews in parallel with a literature review of undergraduate QI. As lessons from the literature review and data on the experiences of students and faculty emerged, it became clear that findings from the literature review resonated with data about the experiences of our students and faculty. Some of the challenges experienced by staff and students could have been anticipated and avoided if the literature review had informed both curriculum design and implementation processes.
A key concern within the medical school was the sustainability of the module in terms of its scale and scope. This proved to be an issue. Students and supervisors reported that identifying and completing a QIP took much more time than that allocated within the module design. There were other logistical problems. Students started their QIP in their first rotation, then moved to their next rotation. This meant students had to travel back to the QIP site to collect data. For some, the travel time and distance was significant. Not being at the QIP site created problems with arranging access to the necessary data. This proved disruptive in other unintended ways. There were clashes between planned teaching within the second rotation and the need to collect QIP data. As students fulfilled their commitments to their QIP they were missing other teaching.
In clinical settings, there appeared to be confusion around what students described as ‘QI and KCL QI’. Unpacking this, it appeared that the students were navigating the expectations of their QI supervisors who had experience of QI but who were not familiar with the range of module requirements. A more serious issue for students was when their supervisors had little or no experience of QI.
Students described some projects being easier than others and some taking more time, and some supervisors being ‘better’ than others. From a curriculum perspective, this identified a need to ensure a more equitable student experience. The attachment of the education performance measure (EPM) to assessment intensified student concerns about assessment.
The EPM measures clinical and non-clinical skills, knowledge and performance of students during their undergraduate medical degree. It is important for medical students because it forms part of their application to foundation programmes. The measure contributes to their overall score which has implications for which foundation school they are accepted for. It contributes to a highly competitive process and one that is consequential for students.
In summary, the student experience lay along a continuum that at one end had QIP projects that made a significant improvement within trusts and primary care. These projects were presented in their clinical settings and at national and international conferences, and were published. At the other end of the continuum, students struggled to find an actionable project or have early and regular communication with their supervisors.
Supervisors' experiences mirror some of the issues raised by students. They felt there was insufficient time in their job plan to support undergraduate QIPs. This created a tension between service commitments and finding time to teach. There was confusion over assessments and concerns around consistency and fairness. There was an initial confusion about whether assessment focused upon ‘having a successful project’ (implying a project with a successful outcome and positive impact contributing to or informing improvements in service) or whether assessment focused upon what students had learned about the methodology of QI, irrespective of the outcome of the project itself.
Supervisors identified challenges they faced in supporting undergraduate QIPs and how their education role could be recognised by the medical school in ways that contributed to their annual appraisals and portfolios.
Those supervisors who were experienced QI leads within their clinical setting found it easier to identify projects for students or elements within QIPs that were feasible for undergraduates to undertake and from which they would benefit.
There was broad agreement that those who ‘hit the ground running’ and avoided delays setting up their project were able to complete on time. Delays during start-up would later impact on student workload and potential for success. Failure to begin the QIP and progress it within the first 3 months led to what some students called ‘post-January panic’.
Organising meetings with [our] supervisors, created delays. [It] took about 2 or 3 months for the initial planning stage [to be completed]. [By January we] were on a different site. [That] makes it hard to [set up] meetings and getting emails sorted out [to plan access to and collect data].
Projects that had low recruitment numbers or were difficult to shape into plan, do, study, act (PDSA) cycles created difficulty for students.
So, in the 2 or 3 months of the project, we would just have way too few patients to be able to generate anywhere near the data that would be necessary for a single PDSA cycle, let alone several. So, we had to rebuild that project with that in mind.
Many students describe feeling unprepared to undertake a QIP, nor did they think the introduction to the module was realistic about the kind of challenges that they might encounter.
I think with us, [we were] thrown into it. You are sitting there trying to figure out in the first few months: what is a PDSA cycle, what is a fishbone diagram, how do run-charts work?
There was much anxiety about the module assessment. Despite reassurances during the module introduction that that they would be assessed on their learning about the QI process, many felt that was not the case.
‘No, don't worry, it's about the learning,’ we were reassured. We [the group] thought fine, we've actually learnt quite a lot. But when it came to our marks and my feedback, it literally said: ‘You did not show positive results,’ and I was just like, ‘What?’
A highly valued ‘unanticipated’ learning for students was coming to a better understanding of the dynamics of clinical settings and teams and how to manage these in relation to progressing projects.
There's just a lot of like human factors [in QI projects]. You're not going to change [things] overnight.
I feel I can probably negotiate hostile environments better than I could before.
Student themes are summarised in Table 1.
Many of the issues raised by students were also raised by faculty. These relate to how the curriculum was implemented.
A common concern within faculty was a lack of time in their job plans to supervise students.
We feel really stretched clinically, so having dedicated time [with] students is a real concern going forward.
Faculty talked of a lack of clarity about the requirements of the module.
This is the kind of thing that, if you got everyone in a room at the beginning and explained what they had to do, rather than getting supervisors to read a relatively long document, that might be more helpful.
Faculty talked about the importance of reward and recognition for their role as supervisors.
A huge chunk of appraisal is all-around quality improvement activity. If you are able to get some mutual gain for your department out of [supervising a QIP], it makes more sense that you're spending time on that.
A summary of themes from faculty can be found in Table 2.
The literature review highlighted that many of the challenges encountered by staff and students could have been anticipated and potentially avoided.
There has been an enduring lack of QI capacity in clinical environments of healthcare professionals with appropriate QI methodology experience.4
A need for clarity about and ease of conducting a QIP was identified in the literature as a key enabler supporting student engagement. For example, a clear overview and timeline of a QIP with readily available access to databases and example projects was beneficial.5 The importance of institutional support for reward and recognition was also highlighted in the literature.4,6
Integrating QI methodology across the curriculum was recognised as an important enabling factor, with broad agreement that teaching should begin in the early pre-clinical years and continue throughout training. 7
There are many more examples where the literature could helpfully have informed the design and implementation of this module for the benefit of students and faculty. Such was the pressure to create and launch a new curriculum that reviewing the scholarship of teaching in this area was overlooked (Table 3).
Discussion
A focus on rapid design and implementation potentially compromised our educational responsibility. This compels us to reflect upon, reconsider and improve our approach.
It is important to note that when implementing a new curriculum, it is common for there to be ‘teething problems’ and unforeseen logistical issues.
This study has some limitations. Data collection happened at the end of the QI module creating time constraints for recruiting students. There were 400 students in their penultimate year, the sample recruited is small. Time to recruit a larger and more stratified sample would have been desirable.
Though both the IHI Open School Quality Improvement Practicum and the QI module are clear that engaging undergraduates in QI is to develop their learning and prepare them to better undertake QI in practice, this was not what many of the students interviewed believed.
Changes have subsequently been introduced that align with advice from the literature and improve student and faculty experience. For example, students are taught QI methodology in their early years, initiatives have built capacity for QI supervisors and faculty development is provided for those supervising on the module. This includes clarification around assessment.
The disconnect between the design and implementation of the module and lessons from the literature, for us, opened questions about what the relationship between scholarship and curriculum development was. This raised some key questions.
How much is pedagogic scholarship really valued and understood?
Are there deeper issues to explore around discipline-based educational values and culture and how these influence our learning and teaching processes?
A fundamental value in clinical care is to do no harm. The rush to develop and implement a QI module compromised this fundamental value.
While the evaluation informed positive changes to the module, a decision was taken to not continue the evaluation in future years. The rationale was that further work was unlikely to find anything new.
Part of fulfilling our educational responsibility is to build an increasingly persuasive case for the integration of scholarship, curriculum design, learning and teaching. This will require engaging and supporting clinical teachers to relate scholarship to the practical task of curriculum design and implementation. That will require faculty development.
We draw attention to some of the avoidable challenges that faculty and students faced and some of the benefits of undergraduate QIPs for students and clinical faculty. The journey of increasing understanding and confidence in the role scholarship can play, will involve navigating different worldviews and priorities. This is a duty of care we have in order to provide the best possible educational experience for students and faculty.
To achieve this goal will require education leaders and educators to develop a nuanced understanding of the values, culture and systems factors that shape our assumptions, approaches and educational practices.
This study challenged us to reconsider how the relationship between scholarship and the curriculum is enacted and the implications for student and faculty experiences.
- © Royal College of Physicians 2021. All rights reserved.
References
- ↵
- Madigosky W
- ↵
- Partlett M
- ↵
- Greenlaugh T
- ↵
- Davis NL
- ↵
- ↵
- ↵
- Gonzalo JD
Article Tools
Citation Manager Formats
Jump to section
Related Articles
- No related articles found.
Cited By...
- No citing articles found.