Development of a web-based training programme for respiratory physicians in Yorkshire
Abstract
This article describes the establishment of a web-based training programme for postgraduate respiratory trainees. A pilot survey assessed trainees’ experiences of online learning. All trainees had used the internet for work-related purposes with commonly used resources being Google, national society websites and specialty journals. A modular training programme based on article review, multiple choice questions (MCQs) and a reflective feedback exercise was subsequently established and trainees’ experiences reviewed at three and 15 months. Trainees found that topics covered were appropriate, contained novel information and subjectively impacted on practice. Common problems were technical difficulties accessing the site (37%) and journal articles (74%), ambiguity of MCQs and difficulty finding time during normal working hours to complete modules. Trainees felt that feedback from MCQs and discussion with colleagues and their educational supervisor were useful methods to support their learning, but did not find the self-reflection exercise helpful.
Introduction
The introduction of Modernising Medical Careers (MMC) has significantly altered postgraduate medical training in the UK and there is added pressure to deliver training more effectively and efficiently. Respiratory medicine encompasses a wide range of clinical conditions and the new curriculum emphasises the importance of work-based learning.1 Theoretical training will continue to be required and web-based training (WBT) is one of the modalities by which this may be delivered. WBT is the provision of specific training requirements through an online medium and has the potential to provide a structure for theoretical knowledge and the ability to help trainees assess their own knowledge base, reflect on their understanding and plan future study.
Two benefits of WBT are that it can be delivered to a geographically dispersed population, and that materials can be accessed by the trainee at the time of their choosing. Respiratory medicine placements in Yorkshire cover a large geographical area and attendance at a training day in Leeds can involve a round trip of over 130 miles. Registrars on partial-shift rotas, which involve daytime working on medical assessment units and night shifts, are potentially unavailable for up to 30% of training days. This is a similar figure to that reported by other groups.2 If training materials could be accessed online at the time and place of the trainees choosing this could reduce the requirement to attend other aspects of the programme. It has been demonstrated that groups of healthcare professionals will accept computer-based training in the workplace.3 Other potential advantages include the ability to link resources in many different formats and to encourage more independent and active learning.4 WBT can support trainee feedback,2 an essential part of adult learning.5 Permanent records allow trainees to return later and reflect on material, supporting the construction of reflective accounts.6 Online resources can be accessed on demand allowing immediate or delayed repetition and self-testing.7 Online discussions allow a greater range of perspectives and ideas to be dispersed to students.8
Potential problems with this approach include technical issues such as accessibility, a trainee's level of information technology (IT) literacy, and site content and structure. Volume and unreliability of information is also an issue and trainees must learn the skills sufficient to access and use the internet effectively.7 Training should continue to be student centred and not rely solely on novel technologies.9 Online learning can also prove to be expensive in terms of time required to set up systems and ongoing costs to maintain and update them.2,3 From an educational perspective, systems should support deep-learning styles with the aim of changing long-term practice.5 WBT does not stand alone and a blended approach, with the integration of different resources and activities within a range of learning environments, is recommended.10
Methods
In April 2007, opportunities for respiratory training in Yorkshire were 10 face-to-face training days, with trainees expected to attend eight days per year. This was becoming increasingly difficult due to the pressures of new working rotas. It was decided to establish a WBT programme which could deliver a further two days of recognised training to give more flexibility. An initial pilot study was circulated to trainees to assess if this style of training would be acceptable to trainees and their preferred styles of online training.
On the basis of these results the first WBT modules became available for online completion in June 2007. Modules were prepared by the specialty training committee (STC) and uploaded to a section of the Yorkshire Deanery website by the IT manager. There were no direct IT costs for the respiratory training committee for hosting the site. Further surveys were circulated to trainees after three months and again after 15 months once the first training module had been added to assess experiences of using WBT.
Results
Pilot study
In total, 26 of the 34 trainees returned the initial survey (76%). Of these, 25 had a broadband internet connection at home (96%). All trainees reported using the internet for work purposes and over 90% had used it for this reason in the preceding seven days. Popular websites at that time included Google (92% had accessed it in the preceding seven days), the British Thoracic Society website (77%), www.doctors.org.uk (62%) and various respiratory medicine journals (62%) (Fig 1). Blogs and chat rooms were rarely used with only 19% of trainees having accessed these for work reasons in the previous 12 months. Preferred styles of online learning were the review of photographs with questions, case reports and questions on blind cases. Blogs and online discussion via chat rooms were considered to be less useful (Fig 2). Trainees felt that the main benefits of an online learning programme were that it could be done in their own time (85%), at home (77%) and that questions or modules could be repeated (81%). It was noted that two trainees could see no benefits to online training packages, which supports the concept of a blended approach to training.
Initial experiences
Following the pilot study it was felt that there was general support among the trainees for an online learning programme. It was considered to be technically difficult to base modules on complex respiratory cases, and it was initially decided to develop them based on a review of relevant journal articles. Consultants in the region with a declared interest in the subspecialty topic selected articles and prepared associated multiple choice questions (MCQs) and feedback to support each article. A self-reflection questionnaire was completed following each module. Trainees discussed the results of the MCQs and reflective feedback exercise with their educational supervisor. Completion of a module credited the trainee with one hour of training. The aim was to upload one new module per month, to offer 12 hours of training in this way per year, equating to two whole days.
The trainees’ initial experiences were assessed at three months following the uploading of the first module in September 2008. Responses were received from 27 of 40 trainees (68%). Eleven trainees had not used the site (41%). Of those who had, a median of two modules had been completed, taking an average time of 59 minutes (range 30–120 min). The most common times for completing the modules were during protected teaching time (33%) and at home in the evening (28%) or at the weekend (22%) (Fig 3). Technical difficulties encountered included problems navigating to the site (31%) and an inability to access journal articles (25%) (Fig 4).
All trainees felt that the topics chosen were appropriate and stated that they learnt something from the modules. Sixty-nine per cent felt they would change aspects of their practice and 25% did further reading on topics of interest. Trainees considered that the most useful educational aspect of the site was the feedback received from the MCQs, followed by discussion of articles with colleagues, discussion of modules with their educational supervisor and, finally, self-reflection using the reflective feedback template on the site. Suggestions for future development were use of an online forum to facilitate discussion and recording of the MCQ answers to allow future review.
This feedback demonstrated the need to further engage trainees with online learning. Trainees were able to complete articles during their protected teaching time and some accessed materials during nights on call demonstrating the flexibility of this method of learning. It was felt that technical issues would reduce as familiarity with the site increased. The average time completing each article was 59 minutes. As each module credited trainees with one hour of training this suggested that the articles and MCQs were of an appropriate difficulty level. The reflective feedback exercise was generally not felt to be helpful – 44% scored this as the least useful aspect of the site. It was interesting that trainees suggested establishing an online forum to facilitate discussion as the original survey showed that chat rooms were one of the least popular methods of online learning.
One year on
Following the review of initial experiences of the site it was decided to make WBT a compulsory element of the training in the region prior to the Annual Review of Clinical Progression (ARCP) in June 2009. It was also suggested that chosen topics should be more explicitly linked to the respiratory curriculum and the face-to-face training days. It was not possible to integrate an online discussion forum as the STC did not have direct control over the software associated with the site and no one was available with the sufficient training or time to moderate discussions.
Questionnaires were sent out in October 2009 and 23 out of 42 responses were received (55%). Three trainees had not used the WBT (two first-year trainees who had only been in post a short period of time and one third-year trainee). At this point 11 modules were available with trainees completing a median of eight (range 1–11). Average time to complete a module had increased slightly to 74 minutes (range 15–150). Fewer trainees were completing modules in their protected teaching time compared to in September 2008 (Fig 3). Responses regarding technical difficulties with the site were similar to those previously seen, with the main issues being problems finding the website (37%) and accessing journals (74%) (Fig 4).
As before, trainees felt topics were appropriate, that they learned something from the modules and that the difficulty level of the MCQs was about right. The trainees are split on whether they would benefit from more training in adult learning, with 35% agreeing it would be helpful, 45% disagreeing and 20% having no opinion. The majority felt there was a good link with topics in the respiratory curriculum (70% agreed or strongly agreed) but very few thought there was good integration with topics in the face-to-face training days (68% disagreed or strongly disagreed). Trainees were asked if the way they used the e-learning had changed after it was made compulsory; 53% agreed or strongly agreed and 32% disagreed or strongly disagreed. Self-reflection was again felt to be the least useful aspect of the learning, with feedback from the MCQs being seen as most useful. A number of opportunities to provide free-text comments regarding the training existed, which also provided useful information (Table 1).
Discussion and summary of experiences
Technical aspects
The STC had difficulty developing the WBT as it did not have direct access to the software on which the training was hosted. In both follow-up surveys trainees suggested the establishment of an online discussion forum to facilitate reflection on the module. This was not, however, possible as the website is controlled by the Yorkshire Deanery. Although this approach made the WBT very cheap to set up, it has limited the ability to develop the training further. Time is required to support WBT and although formal data have not been collected, trainers in the region have reported taking between four and six hours to prepare each module. There were issues regarding accessibility to the site and difficulty navigating to it which did not improve with time.
Access to journal articles was noted to be a problem. For copyright reasons it is not possible to have articles available directly on the site so trainees have to access them via the journal website or local library services. The ability to effectively utilise library services is an important skill for trainees. The articles referenced are generally from large circulation international respiratory journals which should be available to all hospital libraries. In the pilot study, 62% of trainees had accessed a respiratory journal online in the previous seven days and 96% in the previous three months.
There were comments from trainees regarding the quality of MCQs produced. The feedback from the answers appeared to be of sufficient detail in most cases but the trainees often felt the questions themselves were ambiguous. This reflects the difficulty in producing high quality MCQs in general. It is imperative that those involved in creating modules have been on relevant courses to ensure that they know how to produce effective questions. There may be a greater need to standardise the format of questions in the future, but this will depend on further feedback from trainees.
The number of trainees reporting insufficient time to complete modules increased from 6% in September 2008 to 32% in October 2009. One possible explanation would be the change in working patterns to make rotas compliant with the European Working Time Directive which was fully implemented in August 2009. This may also explain the observation that fewer trainees were completing the modules during protected education time in the second follow-up survey. This is a general problem with all training methods and highlights the importance of maximising all training opportunities. There is a need to ensure that trainees receive protected teaching time as part of their work patterns and that this is enforced by educational supervisors.
Educational aspects
Feedback from trainees regarding the WBT was generally positive and suggested that modules were influencing trainee's knowledge and practice, although this evidence is all subjective. Some trainees were able to give specific examples of situations where patient management had been altered following completion of a WBT module. The feedback suggested the topics chosen were appropriate; they were linked to the respiratory curriculum but there is a need to have better integration between the face-to-face and online training in the region. This is a matter for the STC, to ensure that modules on appropriate topics are commissioned. A consultant lead for online training (JK) has been nominated following the initiation of the WBT.
Self-reflection is considered to be an important part of adult learning, but the feedback suggests it is an underutilised tool. It was not considered to be helpful, however, trainees were keen to discuss the modules in groups. This is a way of facilitating self-reflection and is an opportunity to clarify issues. Trainees were split in their opinions on whether further training would improve their experiences of the site. Some elements of self-reflection are addressed in the Yorkshire Deanery core module which is a compulsory course for all trainees in the region, but further training in this skill for all trainees is encouraged.
Finally, the involvement of individuals with specialist training in medical education to guide the development of all novel training methods is encouraged to ensure that they have a sound underlying educational philosophy.
- © 2010 Royal College of Physicians
References
- ↵
- Joint Royal Colleges of Physicians Training Board
- ↵
- ↵
- Columbine AM,
- Wharrad HJ
- ↵
- McKimm J,
- Jollie C,
- Cantillon P
- ↵
- Fry H,
- Ketteridge S,
- Marshall S.
- ↵
- ↵
- ↵
- ↵
- ↵
- Littlejohn A,
- Pegler C.
Article Tools
Citation Manager Formats
Jump to section
Related Articles
- No related articles found.
Cited By...
- No citing articles found.