Utilization-focused evaluation.

Context For nearly 40 years, outcome-based models have dominated programme evaluation in health professions education. However, there is increasing recognition that these models cannot address the complexities of the health professions context and studies employing alternative evaluation approaches that are appearing in the literature. A similar paradigm shift occurred over 50 years ago in the broader discipline of programme evaluation. Understanding the development of contemporary paradigms within this field provides important insights to support the evolution of programme evaluation in the health professions.

Methods  In this discussion paper, we review the historical roots of programme evaluation as a discipline, demonstrating parallels with the dominant approach to evaluation in the health professions. In tracing the evolution of contemporary paradigms within this field, we demonstrate how their aim is not only to judge a programme’s merit or worth, but also to generate information for curriculum designers seeking to adapt programmes to evolving contexts, and researchers seeking to generate knowledge to inform the work of others.

Discussion  From this evolution, we distil seven essential elements of educational programmes that should be evaluated to achieve the stated goals. Our formulation is not a prescriptive method for conducting programme evaluation; rather, we use these elements as a guide for the development of a holistic ‘programme of evaluation’ that involves multiple stakeholders, uses a combination of available models and methods, and occurs throughout the life of a programme. Thus, these elements provide a roadmap for the programme evaluation process, which allows evaluators to move beyond asking whether a programme worked, to establishing how it worked, why it worked and what else happened. By engaging in this process, evaluators will generate a sound understanding of the relationships among programmes, the contexts in which they operate, and the outcomes that result from them.

https://onlinelibrary.wiley.com/doi/pdf/10.1111/medu.12091

Rethinking programme evaluation in health professions education: beyond 'did it work?'.

This paper reviews evidence regarding change in health-care provider behaviour and maternal and neonatal outcomes as a result of emergency obstetric and neonatal care (EmONC) training A refined version of the Kirkpatrick classification for programme evaluation was used to focus on change in efficiency and impact of training (levels 3 and 4) Twenty-three studies were reviewed – five randomised controlled trials, two quasi-experimental studies and 16 before-and-after observational studies Training programmes had all been developed in high-income countries and adapted for use in low- and middle-income countries Nine studies reported on behaviour change and 13 on process and patient outcomes Most showed positive results Every maternity unit should provide EmONC teamwork training, mandatory for all health-care providers The challenges are as follows: scaling up such training to all institutions, sustaining regular in-service training, integrating training into institutional and health-system patient safety initiatives and ‘thinking out of the box' in evaluation research

What is the impact of multi-professional emergency obstetric and neonatal care training?

Context Despite increasing numbers of publications, science often fails to significantly improve patient care. Clinician-scientists, professionals who combine care and research activities, play an important role in helping to solve this problem. However, despite the ascribed advantages of connecting scientific knowledge and inquiry with health care, clinician-scientists are scarce, especially amongst non-physicians. The education of clinician-scientists can be complex because they must form professional identities at the intersection of care and research. The successful education of clinician-scientists requires insight into how these professionals view their professional identity and how they combine distinct practices. Objectives This study sought to investigate how recently trained nurse- and physiotherapist-scientists perceive their professional identities and experience the crossing of boundaries between care and research. Methods Semi-structured interviews were conducted with 14 nurse- and physiotherapist-scientists at 1 year after they had completed MSc research training. Interviews were thematically analysed using insights from the theoretical frameworks of dialogical self theory and boundary crossing. Results After research training, the initial professional identity, of clinician, remained important for novice clinician-scientists, whereas the scientist identity was experienced as additional and complementary. A meta-identity as broker, referred to as a ‘bridge builder’, seemed to mediate competing demands or tensions between the two positions. Obtaining and maintaining a dual work position were experienced as logistically demanding; nevertheless, it was considered beneficial for crossing the boundaries between care and research because it led to reflection on the health profession, knowledge integration, inquiry and innovation in care, improved data collection, and research with a focus on clinical applicability. Conclusions Novice clinician-scientists experience dual professional identities as care providers and scientists. The meta-position of being a broker who connects care and research is seen as core to the unique clinician-scientist identity. To develop this role, identity formation and boundary-crossing competencies merit explicit attention within clinician-scientist programmes.

https://onlinelibrary.wiley.com/doi/pdf/10.1111/medu.13241

Professional identity in clinician-scientists: brokers between care and science.

Introduction In an effort to increase the rigour of evaluation in health professions education (HPE), a range of evaluation approaches are used. These largely focus on outcome evaluation as opposed to programme evaluation. We aim to review and critique the use of outcome evaluation models, using the Kirkpatrick Model as an example given its wide acceptance and use, and advocate for the use of programme evaluation models that help us understand how and why outcomes are occurring. Methods We systematically searched OVID medline, Scopus, CINAHL and Pubmed, and hand searched six leading HPE journals to provide an overview of the use of the Kirkpatrick Model as well as a range of programme evaluation models in HPE. In addition to this, we synthesised the existing critiques of the Kirkpatrick Model as an example of outcome evaluation, to highlight the limitations of such models. Results The use of the Kirkpatrick Model in HPE is widespread and increasing; however, studies focus on categorising outcomes, rather than explaining how and why they occur. The main criticisms of the model are as follows: it is outcomes focused and fails to consider factors that can impact training outcomes; it assumes positive casual linkages between the levels; there is an assumption that the higher-level outcomes are more important; and unintended impacts are not considered. The use of the Kirkpatrick Model by the MERSQI, BEME and WHO contribute to the myth that the Kirkpatrick Model is the gold standard for programme evaluation. Discussion Moving forward, evaluations of HPE interventions must shift from focusing largely on measuring outcomes of interventions with little consideration for how and why these outcomes are occurring to programme evaluation that investigates what contributes to these outcomes. Other models that facilitate the evaluation of the complex processes that occur in HPE should be used instead of Kirkpatrick's.

Evaluation in health professions education-Is measuring outcomes enough?

PurposeTo explore how the Canadian Child Health Clinician Scientist Program (CCHCSP) works to achieve prearticulated and emergent outcomes.MethodIn 2009, after gaining ethical approval from the Hospital for Sick Children, the authors examined quantitative data (e.g., participation in curricu

Going beyond Kirkpatrick in evaluating a clinician scientist program: it's not "if it works" but "how it works".

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Gwen Burrows

Author Tools

Chat about Author

Papers

Going beyond Kirkpatrick in evaluating a clinician scientist program: it's not "if it works" but "how it works".