CONTEXT Situational judgement tests (SJTs) are widely used to evaluate 'non-academic' abilities in medical applicants. However, there is a lack of understanding of how their predictive validity may vary across contexts. We conducted a systematic review and meta-analysis to synthesise existing evidence relating to the validity of such tools for predicting outcomes relevant to interpersonal workplace performance. METHODS Searches were conducted in relevant databases to June 2019. Study quality and risk of bias were assessed using the Quality In Prognosis Studies (QUIPS) tool. Results were pooled using random effects meta-analysis and meta-regressions. RESULTS Initially, 470 articles were identified, 218 title or abstracts were reviewed, and 44 full text articles were assessed with 30 studies meeting the final inclusion criteria and were judged, overall, to be at moderate risk of bias. Of these, 26 reported correlation coefficients relating to validity, with a pooled estimate of 0.32 (95% confidence interval 0.26 to 0.39, P < .0001). Considerable heterogeneity was observed (I2  = 96.5%) with the largest validity coefficients tending to be observed for postgraduate, rather than undergraduate, selection studies (β = 0.23, 0.11 to 0.36, P < .001). The correction of validity coefficients for attenuation was also independently associated with larger effects (β = 0.13, 0.03 to 0.23, P = .01). No significant associations with test medium (video vs text format), cross-sectional study design, or period of assessment (one-off vs longer-term) were observed. Where reported, the scores generally demonstrated incremental predictive validity, over and above tests of knowledge and cognitive ability. CONCLUSIONS The use of SJTs in medical selection is supported by the evidence. The observed trend relating to training stage requires investigation. Further research should focus on developing robust criterion-relevant outcome measures that, ideally, capture interpersonal aspects of typical workplace performance. This will facilitate additional work identifying the optimal place of SJTs within particular selection contexts and further enhancing their effectiveness.

https://onlinelibrary.wiley.com/doi/epdf/10.1111/medu.14201

Situational judgement test validity for selection: A systematic review and meta-analysis.

What subjects UK medical schools teach, what ways they teach subjects, and how much they teach those subjects is unclear. Whether teaching differences matter is a separate, important question. This study provides a detailed picture of timetabled undergraduate teaching activity at 25 UK medical schools, particularly in relation to problem-based learning (PBL). The Analysis of Teaching of Medical Schools (AToMS) survey used detailed timetables provided by 25 schools with standard 5-year courses. Timetabled teaching events were coded in terms of course year, duration, teaching format, and teaching content. Ten schools used PBL. Teaching times from timetables were validated against two other studies that had assessed GP teaching and lecture, seminar, and tutorial times. A total of 47,258 timetabled teaching events in the academic year 2014/2015 were analysed, including SSCs (student-selected components) and elective studies. A typical UK medical student receives 3960 timetabled hours of teaching during their 5-year course. There was a clear difference between the initial 2 years which mostly contained basic medical science content and the later 3 years which mostly consisted of clinical teaching, although some clinical teaching occurs in the first 2 years. Medical schools differed in duration, format, and content of teaching. Two main factors underlay most of the variation between schools, Traditional vs PBL teaching and Structured vs Unstructured teaching. A curriculum map comparing medical schools was constructed using those factors. PBL schools differed on a number of measures, having more PBL teaching time, fewer lectures, more GP teaching, less surgery, less formal teaching of basic science, and more sessions with unspecified content. UK medical schools differ in both format and content of teaching. PBL and non-PBL schools clearly differ, albeit with substantial variation within groups, and overlap in the middle. The important question of whether differences in teaching matter in terms of outcomes is analysed in a companion study (MedDifs) which examines how teaching differences relate to university infrastructure, entry requirements, student perceptions, and outcomes in Foundation Programme and postgraduate training.

/pdf/the-analysis-of-teaching-of-medical-schools-atoms-survey-an-4s6v1w45wk.pdf

The Analysis of Teaching of Medical Schools ( AToMS ) survey: an analysis of 47,258 timetabled teaching events in 25 UK medical schools relating to timing, duration, teaching formats, teaching content, and problem-based learning

This work was unfunded but the ideas stem from numerous funded projects from over the years, particularly from the General Medical Council (GMC) and Medical Schools Council (MSC) of the UK Our thanks also to the anonymous reviewers of an earlier version of this paper for their helpful feedback

https://onlinelibrary.wiley.com/doi/pdf/10.1111/medu.13670

Thinking of selection and widening access as complex and wicked problems

Objectives Many countries are driving forward policies to widen the socioeconomic profile of medical students and to train more medical students for certain specialties. However, little is known about how socioeconomic origin relates to specialty choice. Nor is there a good understanding of the relationship between academic performance and specialty choice. To address these gaps, our aim was to identify the relationship between socioeconomic background, academic performance and accepted offers into specialty training. Design Longitudinal, cohort study using data from the UK Medical Education Database (https://www.ukmed.ac.uk/). Participants 6065 (60% females) UK doctors who accepted offers to a specialty training (residency) post after completing the 2-year generic foundation programme (UK Foundation Programme) between 2012 and 2014. Main outcome measures Χ2 tests were used to examine the relationships between sociodemographic characteristics, academic ability and the dependent variable, specialty choice. Multiple data imputation was used to address the issue of missing data. Multinomial regression was employed to test the independent variables in predicting the likelihood of choosing a given specialty. Results Participants pursuing careers in more competitive specialties had significantly higher academic scores than colleagues pursuing less competitive ones. After controlling for the presence of multiple factors, trainees who came from families where no parent was educated to a degree level had statistically significant lower odds of choosing careers in medical specialties relative to general practice (OR=0.78, 95% CI, 0.67 to 0.92). Students who entered medical school as school leavers, compared with mature students, had odds 1.2 times higher (95% CI, 1.04 to 1.56) of choosing surgical specialties than general practice. Conclusions The data indicate a direct association between trainees’ sociodemographic characteristics, academic ability and career choices. The findings can be used by medical school, training boards and workforce planners to inform recruitment and retention strategies.

https://bmjopen.bmj.com/content/bmjopen/9/3/e026961.full.pdf

Relationship between sociodemographic factors and specialty destination of UK trainee doctors: a national cohort study.

Objectives Currently relative performance at medical school (educational performance measure (EPM) decile), additional educational achievements and the score on a situational judgement test (SJT) are used to rank applicants to the UK Foundation Years postgraduate medical training programme. We sought to evaluate whether these three measures were predictive of subsequent successful completion of the programme, and thus were valid selection criteria. Methods Data were obtained from the UK Medical Education Database (UKMED) on 14 131 UK applicants to the foundation programme starting in 2013 and 2014. These data included training outcomes in the form of Annual Reviews of Competency Progression (ARCPs), which indicated whether the programme was successfully completed. The relationship between applicants’ performance on the three selection measures to the odds of successful programme completion were modelled. Results On univariable analyses, all three measures were associated with the odds of successful completion of the programme. Converting the SJT score to deciles to compare the effect sizes suggested that one decile increase in the EPM increased the odds of completing the programme by approximately 15%, whereas the equivalent value was 8% for the SJT scores. On multivariable analyses (with all three measures included in the model), these effects were only independently and statistically significant for EPM decile (OR 1.14, 95% CI 1.10 to 1.18, p Conclusions The EPM decile and SJT scores may be effective selection measures for the foundation programme. However, educational achievements does not add value to the other two measures when predicting programme completion. Thus, its usefulness in this context is less clear. Moreover, our findings suggest that the weighting for the EPM decile score, relative to SJT performance, should be increased.

https://bmjopen.bmj.com/content/bmjopen/8/7/e021918.full.pdf

Evaluating the validity of the selection measures used for the UK's foundation medical training programme: a national cohort study.

Educating doctors is expensive and poor performance by future graduates can literally cost lives. Whilst the practice of medicine is highly evidence based, medical education is much less so. Research on medical school selection, undergraduate progression, Fitness to Practise (FtP) and postgraduate careers has been hampered across the globe by the challenges of uniting the data required. This paper describes the creation, structure and access arrangements for the first UK-wide attempt to do so. A collaborative approach has created a research database commencing with all entrants to UK medical schools in 2007 and 2008 (UKMED Phase 1). Here the content is outlined, governance arrangements considered, system access explained, and the potential implications of this new resource discussed. The data currently include achievements prior to medical school entry, admissions tests, graduation point information and also all subsequent data collected by the General Medical Council, including FtP, career progression, annual National Training Survey (NTS) responses, career choice and postgraduate exam performance data. UKMED has grown since the pilot phase with additional datasets; all subsequent years of students/trainees and stronger governance processes. The inclusion of future cohorts and additional information such as admissions scores or bespoke surveys or assessments is now being piloted. Thus, for instance, new scrutiny can be applied to selection techniques and the effectiveness of educational interventions. Data are available free of charge for approved studies from suitable research groups worldwide. It is anticipated that UKMED will continue on a rolling basis. This has the potential to radically change the volume and types of research that can be envisaged and, therefore, to improve standards, facilitate workforce planning and support the regulation of medical education and training. This paper aspires to encourage proposals to utilise this exciting resource.

/pdf/the-uk-medical-education-database-ukmed-what-is-it-why-and-4lg3na87x9.pdf

The UK medical education database (UKMED) what is it? Why and how might you use it?

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Thomas Oppé

Author Tools

Chat about Author

Papers

The UK medical education database (UKMED) what is it? Why and how might you use it?