Skip to main content

Main menu

  • Home
  • Our journals
    • Clinical Medicine
    • Future Healthcare Journal
  • Subject collections
  • About the RCP
  • Contact us

Future Healthcare Journal

  • FHJ Home
  • Content
    • Current
    • Ahead of print
    • Archive
  • Author guidance
    • Instructions for authors
    • Submit online
  • About FHJ
    • Scope
    • Editorial board
    • Policies
    • Information for reviewers
    • Advertising

User menu

  • Log in

Search

  • Advanced search
RCP Journals
Home
  • Log in
  • Home
  • Our journals
    • Clinical Medicine
    • Future Healthcare Journal
  • Subject collections
  • About the RCP
  • Contact us
Advanced

Future Healthcare Journal

futurehosp Logo
  • FHJ Home
  • Content
    • Current
    • Ahead of print
    • Archive
  • Author guidance
    • Instructions for authors
    • Submit online
  • About FHJ
    • Scope
    • Editorial board
    • Policies
    • Information for reviewers
    • Advertising

Exception reporting in 2018: how often is it happening?

Matthew Roycroft
Download PDF
DOI: https://doi.org/10.7861/fhj.2019-0059
Future Healthc J October 2020
Matthew Roycroft
ANorthern General Hospital, Sheffield, UK and co-chair of the RCP Trainees Committee, Royal College of Physicians, London, UK
Roles: ST7 geriatric medicine
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: matthew.roycroft@nhs.net
  • Article
  • Figures & Data
  • Info & Metrics
Loading

ABSTRACT

Background Exception reporting is the main process in England to resolve issues related to junior doctor working hours. Concerns have been raised regarding variable report submission, but no significant exploration has occurred. This study assesses frequency of exception reporting and correlates it with frequency of working beyond rostered hours and overall satisfaction.

Methods National training survey (NTS) scores for ‘Overall Satisfaction’ and frequency of working beyond rostered hours was obtained for 26 randomly identified trusts throughout England and correlated with exception reporting frequency from guardian of safe working (guardian) quarterly reports covering April 2018.

Results Guardian reports were obtained for 24 trusts. NTS data suggest trainees worked beyond their rostered hours 12.1 times per quarter (interquartile range (IQR) 10.0–12.9) whereas guardian reports show they exception reported 0.15 times per quarter (IQR 0.084–0.25). Trainees exception report 1.2% of the time they work beyond rostered hours (IQR 0.8–2.4%).

Frequency of exception reporting correlates poorly with the frequency with which trainees work beyond rostered hours (coefficient −0.22) and with a marker of overall satisfaction (coefficient −0.21).

Conclusion The current exception reporting process significantly under-reports trainee working hours although there is regional variation.

KEYWORDS
  • Medical education
  • quality assurance
  • exception reporting
  • working hours
  • wellbeing

Introduction

The guardian of safe working (guardian) role was introduced in England as part of the 2016 junior doctor contract to ‘represent and resolve issues related to working hours for the junior doctors’.1 Digital exception reporting allows instances of either working beyond contracted hours, missing breaks or education opportunities to be reported to the guardian. Most reports are for working beyond contracted hours. The guardian is required to submit a quarterly report to the trust board summarising collated data from these exception reports.2 While there is no requirement on how to present the data, a sample template is available as a guide.3

Despite widespread encouragement from major organisations (including the professional regulator, unions, royal colleges and NHS Employers) for trainees to submit exception reports, it is now widely accepted that exception reporting has only been successful in certain situations or settings and that ‘its overall effectiveness is being restricted by a number of systemic limitations’.4–7 A recent study exploring the topic via freedom of information requests suggested that, since 2015, junior doctors had worked beyond their contracted terms and conditions more than 63,309 times – the count of exception reports submitted.8 In response to this report, the chief executive of NHS Employers (the organisation that acts on behalf of NHS trusts) described exception reporting as ‘working well overall’ but acknowledged ‘there are areas where guardians of safe working hours and doctors tell us improvement is needed’.9 NHS England, in collaboration with major stakeholders, have set up a group to ‘address issues around exception reporting’.7

An early 2017 analysis of exception reporting data within one region of England showed very poor correlation between number of exception reports submitted and frequency with which junior doctors reported working beyond their contracted hours when this was looked at separately.10 The cause was unclear but one purported explanation of this disparity was the immaturity of the system at the time (it had only been around for a year) and that doctors weren't used to submitting reports. It was felt things may settle down with time and as trainees moved between organisations. Suggestions behind variation included fewer incidences of working beyond rostered hours, pressure on trainees not to report, better overall experience or simply trusts/departments that engage better with trainees encouraging them to report more actively. It's unknown whether with maturity the system has improved and whether it now correlates with actual frequency with which trainees work beyond rostered hours and thus whether the number of reports per trainee can be used to compare trusts.

Here, using results from spring 2018, we compare exception reporting frequency (reports per trainee per quarter) across England with trainees self-reported frequency of working beyond rostered hours at a similar point in time when asked anonymously via the General Medical Council's (GMC's) national training survey (NTS).11 We also compare exception reporting frequency with a measure of trainee overall satisfaction of working within that organisation as a first step to assessing whether number of reports suggests high or low organisational performance.

Methods

Study design

Twenty-six employers (healthcare trusts) were randomly identified from throughout England. Efforts were made to obtain quarterly guardian reports for the quarter covering April 2018 for these trusts and calculate from these the number of exception reports per trainee per quarter. For the same trusts, trainee self-reporting of frequency with which they worked beyond rostered hours was obtained from the responses to the anonymous GMC's NTS allowing an approximation of the number of times trainees work beyond rostered hours per trainee per quarter. Correlation coefficients were then calculated comparing the two data sources. A marker of trainee ‘overall satisfaction’ was also obtained for each trust from the NTS and this was correlated with number of exception reports per trainee per quarter.

Trust identification

Two trusts were randomly identified from each of the 13 Health Education England regions of England. Trusts were randomly selected from a list generated by the GMC's NTS filtered by ‘Trust/Board’. Each trust in a region was allocated a number and random.org was used to select two numbers in the region.

Trainee self-reported data

The GMC's 2018 NTS was open between 20 March 2018 to 09 May 2018. Respondents to this survey expect a reasonable degree of anonymity with no results released for any category where there are fewer than three respondents. Results are available online, broken down by trust and individual survey question.11 Results for the question ‘In this post, how often (if at all) have you worked beyond your rostered hours’ along with the number of respondents was extracted by filtering by ‘Trust/Board’ and expanding the ‘Work Load’ domain. Responses had the option of being ‘Daily’, ‘Weekly’, ‘Monthly’, ‘Less than once a month’ and ‘Never’. The number of respondents is given as a five-number range eg 426–430 and the central number in this range was used as the number of respondents.

Frequency with which trainees on average worked beyond their rostered hours was then calculated using a few assumptions: a trainee works 11 weeks in a quarter (allowing for 6 weeks’ annual leave and 2 weeks’ study leave per annum split evenly) and works only Monday–Friday on those weeks. Thus, a trainee reporting daily work beyond rostered hours was assumed to have done this 55 times in a quarter. To approximate the number of times that trainees work beyond rostered hours per trainee per quarter this figure was divided by the number of trainees completing the survey.

A trust score for overall satisfaction was extracted by using the ‘Overall Satisfaction’ domain score after filtering by ‘Trust/Board’. This is a number already calculated from the mix of questions making up the domain with higher values showing better satisfaction than lower.

Guardian of safe working reports

Efforts were made to obtain the quarterly guardian reports covering April 2018 as this was the middle month of the NTS survey. A stepwise approach was followed for obtaining these reports: searching the identified trust website for ‘guardian of safe working’ or ‘guardian’; reviewing all board meeting agendas/minutes from May to August for inclusion of this report; an email request to the secretary of the trust board; and a freedom of information request. If no response was heard, a further email was sent 28 days later. The website searches were done in early–mid-January 2019, the first emails all sent on 24 January 2019 and follow up emails on 21 February 2019 and trusts were given 4 weeks to respond. If a trust redirected an email, the thread was followed up till the report was obtained or they didn't respond to a reminder.

From these reports, the total number of trainees, number of trainees on the 2016 NHS doctors and dentists in training contract (TCS) and the total number of exception reports was extracted from the text, tables or graphs, and estimated from the graphs if no exact figure was given. Piloting showed many reports didn't have the number of exception reports purely for hours (and not for educational reasons) but, in the author's experience, well over 90% of reports are for hours. Reports also didn't often state the number of trainees who could submit exception reports as, while it is a requirement for trainees on the 2016 TCS to be able to submit exception reports, it is highly variable whether trainees not on the 2016 TCS can also do so.12 For calculation purposes it was assumed only those on the 2016 TCS could submit exception reports and that all reports were for hours. Where trusts didn't report number of trainees on the 2016 TCS the number of doctors was estimated by multiplying the number of doctors from the NTS survey by 1.16 (the average of the difference between the NTS number of trainees and those trusts reporting numbers). For the few trusts who reported 4-month ‘quarters’ the number of exception reports was multiplied by 0.75 to allow comparison with other trusts. The number of exception reports per trainee per quarter was calculated from this data.

Statistical analysis

As data were expected to be non-normally distributed, scores for all three areas were ranked and Spearman's rank correlation coefficient was calculated manually using MS Excel comparing reports/trainee/quarter with each of the other two scores. Scatter graphs were created using MS Excel to allow visual assessment of any correlation. As there is significant controversy regarding significance of any individual correlation coefficient no specific cut off for finding correlation was pre-identified.

An estimation of the percentage of times a trainee stays beyond rostered hours that they exception report was calculated by dividing the average number of exception reports/trainee/quarter with the average number of times a trainee stayed beyond rostered hours/trainee/quarter from NTS data. The previously published 63,309 figure for number of exception reports nationally can then be divided by this percentage to give a more accurate approximation of the number of times that trainees on the new contract had actually worked beyond rostered hours in the period they studied (August 2016 – July 2018).

Results

Of the 26 trusts identified, 12 guardian reports were available on the trust website and a further 12 were obtained following email exchanges with the trust. NTS data was obtained for all 26 trusts.

NTS data for all 26 trusts suggested trainees worked beyond their rostered hours an average of 12.1 times per quarter (interquartile range (IQR) 10.0–12.9; range 6.4–20.9). For the 24 trusts for which guardian reports could be obtained, there was little difference: average 11.9 (IQR 9.8–12.8; range 6.4–20.9). Full results are shown in supplementary material S1.

Of the 24 guardian reports obtained, 17 trusts reported the number of doctors on the 2016 TCS, the remaining seven had number of trainees calculated as defined earlier. In total, 1,341 exception reports were recorded. Analysis showed that trainees exception reported an average of 0.15 times per quarter (IQR 0.084–0.25; range 0.030–0.65). Full results are shown in supplementary material S1.

Comparison of the frequency of exception reporting with the frequency with which trainees worked beyond rostered hours is shown in Fig 1a; Spearman's rank correlation coefficient is –0.22. Fig 1b compares the NTS ‘Overall Satisfaction’ domain score with the frequency of exception reporting; Spearman's rank correlation coefficient is –0.21.

Fig 1.
  • Download figure
  • Open in new tab
  • Download powerpoint
Fig 1.

a) Comparison of number of times working beyond rostered hours with number of exception reports. b) Comparison of overall satisfaction with number of exception reports.

On average, trainees submitted exception reports 1.2% of the times that they worked beyond rostered hours (IQR 0.8–2.4; range 0.1–6.5). A more accurate reflection of the number of times trainee doctors have worked beyond rostered hours in the period August 2016–July 2018 is therefore around 5 million.

Discussion

Principal findings

This study suggests that trainees exception report a very small amount (1.2%) of the time they work beyond rostered hours. It shows a lack of correlation (coefficient –0.22) between frequency of exception reporting and frequency with which trainees report staying beyond rostered hours suggesting that, in 2018, frequency of exception reporting cannot be used to compare organisations. It also shows a lack of correlation (coefficient –0.21) between the frequency of exception reporting and a measure of trainee overall satisfaction suggesting that overall satisfaction isn't a key factor in whether or not trainees exception report.

Strengths and weaknesses of the study

This study design has multiple small sources of error and one more significant one. Specifically, it assumes trainee responses to NTS regarding working beyond rostered hours was accurate. Although this is an anonymous survey, it still relies upon accurate recall and has very broad ranges that significantly impact the results. At its worst, a trainee that in reality works beyond rostered hours every other day could respond as it being either daily (thus doubling the frequency) or weekly (which is under half as often). Less significant errors are introduced at various other stages: the guardian reports have slightly different quarters, don't consistently differentiate reports for hours or education, and it is unclear whether trusts count trainees not on the 2016 TCS in their guardian reports. It is unclear in which direction these will affect results. The randomisation method ensures both good geographical spread and also different types and sizes of trusts are represented.

Meaning of the study

This is the first England-wide study looking at frequency of exception reporting and comparing it with trainee self-reported frequency of working beyond rostered hours. The lack of correlation suggests significant local factors influence whether trainees submit reports or not. One of these factors is not trainee overall satisfaction. A recent survey of predominantly surgical trainees showed only 2% reported all unscheduled hours worked and another study of higher specialty physicianly trainees showed only 9% had exception reported in the last year.13,14 These studies looked into why trainees didn't exception report and found similar themes, with many feeling it wasn't of value or wouldn't make a difference, some were actively discouraged or were afraid of reprisals, and some were unaware of the reporting process.

While achieving 100% reporting rate including minor breaches of hours is an ambitious goal, every organisation involved in the process believes trainees who regularly stay beyond rostered hours should exception report every time. Of the trusts identified in this study, NTS data shows 51% of trainees worked beyond their rostered hours either daily or weekly. The suggestion that ‘the [exception reporting] process is working well’ is therefore somewhat questionable as the reporting rate is quite markedly lower than would be expected for these cases.

Unanswered questions and future research

There is little benefit in repeating this study either with data from significantly more trusts or in the next year or two as it is unlikely to show any marked difference. While further exploration of reasons for not exception reporting is possible, the core reasons behind this have already been identified. The major area of work in this area going forward, therefore, needs to be into exploring what measures and techniques improve exception reporting numbers, possibly focusing upon local level changes. This study introduces the idea of calculating the percentage of times a trainee works beyond rostered hours that they exception report and this could be used to set targets for organisational change.

It is possible the current multi-organisational review of the process will introduce some improvement here and re-assessing the situation, after any changes brought by this are embedded, may be worthwhile.

Supplementary material

Additional supplementary material may be found in the online version of this article at www.rcpjournals.org/fhj:

S1 – Fuller dataset for the times when each trainee worked beyond rostered hours.

S2 – Fuller dataset for the exception reports per trainee.

Conflicts of interest

Matthew Roycroft worked for Health Education England in 2017/18.

  • © Royal College of Physicians 2020. All rights reserved.

References

  1. ↵
    1. British Medical Association
    . Guidance for doctors undertaking the guardian of safe working hours role. BMA, 2019. www.bma.org.uk/advice/career/applying-for-a-job/guardian-of-safe-working-hours-role [Accessed 30 September 2019].
  2. ↵
    1. NHS Employers
    . Guidance of guardian of safe working hours quarterly and annual reports. NHS, 2016. www.nhsemployers.org/-/media/Employers/Documents/Need-to-know/Guidance-on-guardian-quarterly-and-annual-reports_November-2016.pdf [Accessed 08 November 2019].
  3. ↵
    1. NHS Employers
    . Quarterly report on safe working hours: doctors and dentists in training (lead employer sample template). NHS, 2017. www.nhsemployers.org/-/media/Employers/Documents/Need-to-know/Template-quarterly-report-for-Lead-Employer-Guardians.docx [Accessed 08 November 2019].
  4. ↵
    1. Kirwan C
    , Lane N, Osanlou O, et al. Exception reporting: A Royal College of Physicians guide. London: Royal College of Physicians, 2017. www.rcplondon.ac.uk/projects/outputs/exception-reporting [Accessed 08 November 2019].
  5. ↵
    1. General Medical Council
    . GMC encourages doctors in training to use exception reports. GMC, 2017. www.gmc-uk.org/news/news-archive/gmc-encourages-doctors-in-training-to-use-exception-reports [Accessed 08 November 2019].
  6. ↵
    1. NHS Employers
    . Building a positive culture around exception reporting. NHS, 2018. www.nhsemployers.org/case-studies-and-resources/2018/12/university-hospitals-of-derby-and-burton—building-a-positive-culture-around-exception-reporting [Accessed 03 September 2020].
  7. ↵
    1. NHS England
    . Exception reporting: joint statement. NHS, 2018 www.england.nhs.uk/publication/exception-reporting-joint-statement [Accessed 08 November 2019].
  8. ↵
    1. Collins A.
    Tens of thousands of junior doctors working beyond contract. HSJ 2019. www.hsj.co.uk/workforce/exclusive-tens-of-thousands-of-junior-doctors-working-beyond-contract/7023993.article [Accessed 08 November 2019].
  9. ↵
    1. Campbell D.
    Junior doctors repeatedly working past shift end, NHS data shows. The Guardian 2019. www.theguardian.com/society/2019/jan/15/junior-doctors-working-past-shift-end-nhs-data-england [Accessed 08 November 2019].
  10. ↵
    1. Roycroft M
    . Early exception reporting data: is it a stick to beat trusts with? [Poster presentation]. York: Future Leaders Conference, 18 January 2018.
  11. ↵
    1. General Medical Council
    . National training surveys. GMC, 2019. www.gmc-uk.org/education/how-we-quality-assure/national-training-surveys [Accessed 08 November 2019].
  12. ↵
    1. NHS Employers
    . FAQs for guardians of safe working hours. NHS, 2017. www.nhsemployers.org/pay-pensions-and-reward/medical-staff/doctors-and-dentists-in-training/information-for-guardians/guardian-faqs-for-guardians [Accessed 08 November 2019].
  13. ↵
    1. Academy Trainees Doctors’ Group
    . ATDG/ASiT TRAIN-ME audit: quantitative analysis of training time – preliminary analysis. Personal communication, 12 September2019.
  14. ↵
    1. Royal College of Physicians, Royal College of Physicians of Edinburgh, Royal College of Physicians and Surgeons of Glasgow
    . Focus on physicians: Census of consultant physicians and higher specialty trainees 2018. London: RCP, 2018. www.rcplondon.ac.uk/projects/outputs/focus-physicians-2018-19-census-uk-consultants-and-higher-specialty-trainees [Accessed 08 November 2019].
View Abstract
Back to top
Previous articleNext article

Article Tools

Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Citation Tools
Exception reporting in 2018: how often is it happening?
Matthew Roycroft
Future Healthc J Oct 2020, 7 (3) 230-233; DOI: 10.7861/fhj.2019-0059

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Exception reporting in 2018: how often is it happening?
Matthew Roycroft
Future Healthc J Oct 2020, 7 (3) 230-233; DOI: 10.7861/fhj.2019-0059
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • ABSTRACT
    • Introduction
    • Methods
    • Results
    • Discussion
    • Supplementary material
    • Conflicts of interest
    • References
  • Figures & Data
  • Info & Metrics

Related Articles

  • Google Scholar

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

  • The investigation and management of pancreatic exocrine insufficiency: A retrospective cohort study
  • Preceptorship scheme for newly qualified physician associates working in general practice in Sheffield
  • Distribution of contemporary sensitivity troponin in the emergency department and relationship to 30-day mortality: The CHARIOT-ED substudy
Show more Original research

Similar Articles

Navigate this Journal

  • Journal Home
  • Current Issue
  • Ahead of Print
  • Archive

Related Links

  • ClinMed - Home
  • FHJ - Home

Other Services

  • Advertising
futurehosp Footer Logo
  • Home
  • Journals
  • Contact us
  • Advertise
HighWire Press, Inc.

Follow Us:

  • Follow HighWire Origins on Twitter
  • Visit HighWire Origins on Facebook

Copyright © 2020 by the Royal College of Physicians