Skip to main content

Main menu

  • Home
  • Our journals
    • Clinical Medicine
    • Future Healthcare Journal
  • Subject collections
  • About the RCP
  • Contact us

Clinical Medicine Journal

  • ClinMed Home
  • Content
    • Current
    • Ahead of print
    • Archive
  • Author guidance
    • Instructions for authors
    • Submit online
  • About ClinMed
    • Scope
    • Editorial board
    • Policies
    • Information for reviewers
    • Advertising

User menu

  • Log in

Search

  • Advanced search
RCP Journals
Home
  • Log in
  • Home
  • Our journals
    • Clinical Medicine
    • Future Healthcare Journal
  • Subject collections
  • About the RCP
  • Contact us
Advanced

Clinical Medicine Journal

clinmedicine Logo
  • ClinMed Home
  • Content
    • Current
    • Ahead of print
    • Archive
  • Author guidance
    • Instructions for authors
    • Submit online
  • About ClinMed
    • Scope
    • Editorial board
    • Policies
    • Information for reviewers
    • Advertising

The independent validation of the Foundation Programme application process: a closer look

Fiona Patterson and Kim Walker
Download PDF
DOI: https://doi.org/10.7861/clinmedicine.16-6-606
Clin Med December 2016
Fiona Patterson
aWork Psychology Group, Derby, UK and visiting researcher, University of Cambridge, Cambridge, UK
Roles: director
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Kim Walker
bUK Foundation Programme Office, Birmingham, UK
Roles: special advisor (recruitment)
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Info & Metrics
Loading

Editor – We highlight four clarifications to Harris and colleagues’ comments about the validity study for the situational judgement test (SJT) for Foundation Programme recruitment.1

Sampling technique and size

Harris et al1 question the 47.7% response; however; this is typical of peer-reviewed published research.2 They also question the sample size, yet n=391 is acceptable for a preliminary study, exceeding many other SJT validation studies;3 it was not practical to obtain whole cohort data. Harris et al query the remedial action sample, which is, by nature, very small; yet we explicitly stated that this should be interpreted cautiously, with further research required.

We agree that a sampling approach in future studies should examine outcomes across whole or several cohorts of foundation year 1 (FY1) doctors, if logistics allow.

Validity of supervisor ratings

Selection research typically uses supervisor performance ratings as outcome data. Harris et al suggest supervisors may not be familiar enough with FY1 doctors to make accurate judgements. However, the overwhelming majority (83.6%) of supervisors were confident in their judgement.

Harris et al question using internal consistency (Cronbach’s alpha), yet this is widely used in questionnaire evaluation.4 Harris et al also suggest items were not independent: two items ‘seem to be testing the same construct’. However, these two items assess behaviours under ‘problem solving and decision making’, so intentionally target the same construct.

Integrity of the study

Harris et al ask ‘should entities who implement new selection tests be responsible for validating them’ and state ‘this work did not undergo external academic peer review’. These statements are inaccurate and misleading. Our study was subject to peer review at every stage of its design, implementation and analysis; overseen by Health Education England, including representatives from the UK Foundation Programme, British Medical Association, Medical Schools Council, General Medical Council, NHS employers, medical and foundation schools, and an independent academic was commissioned to review the report.

Test developers typically publish their own validation studies in addition to inviting external validation studies.

We support Harris et al’s invitation for independent research to examine the validity further. We agree that ongoing research and peer scrutiny is vital to inform continuous improvements.

What does the SJT measure?

It is implied that the SJT may not test relevant constructs. Yet, the study clearly demonstrates the SJT measures key competencies identified in the person specification. There also exists substantial evidence for construct validity of SJTs in many other settings.5,6

Overall, we continue to welcome independent research and peer review to further evaluate the FY1 SJT and we are committed to ongoing improvements.

Conflicts of interest

KW is affiliated with the UK Foundation Programme Office, which oversees the delivery and scoring of the UK Foundation Programme SJT. FP is affiliated with Work Psychology Group, which provides consulting advice and support to Health Education England on selection methodology. Work Psychology Group does not receive any royalties for any methodology used.

  • © Royal College of Physicians 2016. All rights reserved.

References

  1. ↵
    1. Harris BHL
    , Walsh JL, Wilson DJ. The independent validation of the Foundation Programme application process: a closer look. Clin Med 2016;16:92–3.
    OpenUrlFREE Full Text
  2. ↵
    1. Baruch Y
    , Holtom BC. Survey response rate levels and trends in organizational research. Hum Relat 2008;61:1139–60.
    OpenUrlCrossRef
  3. ↵
    1. Bateson JEG
    , Wirtz J, Burke E, Vaughan C. Psychometric sifting to efficiently select the right service employees. Managing Service Quality 2014;24:418–33.
    OpenUrl
  4. ↵
    1. Tavakol M
    , Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ 2011;2:53–5.
    OpenUrlCrossRef
  5. ↵
    1. Koczwara A
    , Patterson F, Zibarras L, et al. Evaluating cognitive ability, knowledge tests and situational judgement tests for postgraduate selection. Med Educ 2012;46:399–408.
    OpenUrlCrossRefPubMed
  6. ↵
    1. Lievens F.
    Adjusting medical school admission: assessing interpersonal skills using situational judgement tests. Med Educ 2013;47:182–9.
    OpenUrlPubMed
Back to top
Previous articleNext article

Article Tools

Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Citation Tools
The independent validation of the Foundation Programme application process: a closer look
Fiona Patterson, Kim Walker
Clinical Medicine Dec 2016, 16 (6) 606-608; DOI: 10.7861/clinmedicine.16-6-606

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
The independent validation of the Foundation Programme application process: a closer look
Fiona Patterson, Kim Walker
Clinical Medicine Dec 2016, 16 (6) 606-608; DOI: 10.7861/clinmedicine.16-6-606
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Sampling technique and size
    • Validity of supervisor ratings
    • Integrity of the study
    • What does the SJT measure?
    • Conflicts of interest
    • References
  • Info & Metrics

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • No citing articles found.
  • Google Scholar

More in this TOC Section

  • Response
  • Functional disorders and chronic pain
  • A further explanation for chest pain without visible coronary artery disease
Show more Letters to the editor

Similar Articles

Navigate this Journal

  • Journal Home
  • Current Issue
  • Ahead of Print
  • Archive

Related Links

  • ClinMed - Home
  • FHJ - Home
clinmedicine Footer Logo
  • Home
  • Journals
  • Contact us
  • Advertise
HighWire Press, Inc.

Follow Us:

  • Follow HighWire Origins on Twitter
  • Visit HighWire Origins on Facebook

Copyright © 2021 by the Royal College of Physicians