TL;DR: Vignettes appear to be a valid and comprehensive method that directly focuses on the process of care provided in actual clinical practice and show promise as an inexpensive case-mix adjusted method for measuring the quality of care given by a group of physicians.
Abstract: ContextBetter health care quality is a universal goal, yet measuring quality
has proven to be difficult and problematic. A central problem has been isolating
physician practices from other effects of the health care system.ObjectiveTo validate clinical vignettes as a method for measuring the competence
of physicians and the quality of their actual practice.DesignProspective trial conducted in 1997 comparing 3 methods for measuring
the quality of care for 4 common outpatient conditions: (1) structured reports
by standardized patients (SPs), trained actors who presented unannounced to
physicians' clinics (the gold standard); (2) abstraction of medical records
for those same visits; and (3) physicians' responses to clinical vignettes
that exactly corresponded to the SPs' presentations.SettingOutpatient primary care clinics at 2 Veterans Affairs medical centers.ParticipantsNinety-eight (97%) of 101 general internal medicine staff physicians,
faculty, and second- and third-year residents consented to be randomized for
the study. From this group, 10 physicians at each site were randomly selected
for inclusion.Main Outcome MeasuresA total of 160 quality scores (8 cases × 20 physicians) were generated
for each method using identical explicit criteria based on national guidelines
and local expert panels. Scores were defined as the percentage of process
criteria correctly met and were compared among the 3 methods.ResultsThe quality of care, as measured by all 3 methods, ranged from 76.2%
(SPs) to 71.0% (vignettes) to 65.6% (chart abstraction). Measuring quality
using vignettes consistently produced scores closer to the gold standard of
SP scores than using chart abstraction. This pattern was robust when the scores
were disaggregated by the 4 conditions (P<.001
to <.05), by case complexity (P<.001), by site
(P<.001), and by level of physician training (P values from <.001 to <.05). The pattern persisted,
although less dominantly, when we assessed the component domains of the clinical
encounter—history, physical examination, diagnosis, and treatment. Vignettes
were responsive to expected directions of variation in quality between sites
and levels of training. The vignette responses did not appear to be sensitive
to physicians' having seen an SP presenting with the same case.ConclusionsOur data indicate that quality of health care can be measured in an
outpatient setting by using clinical vignettes. Vignettes appear to be a valid
and comprehensive method that directly focuses on the process of care provided
in actual clinical practice. Vignettes show promise as an inexpensive case-mix
adjusted method for measuring the quality of care provided by a group of physicians.
TL;DR: This is the first study to establish the performance of administrative data in measuring mental health service provision in a primary care setting and broadly defined administrative measures of mental health have excellent specificity and adequate sensitivity for exploring and understandingmental health service utilization.
Abstract: Objective:We sought to determine the accuracy of administrative data for identifying mental health service provision in primary care.Study Design:This was a chart abstraction study measuring agreement between billing data and clinical data on the binary variable “mental health visit.” Data were coll
TL;DR: In this article, the authors conducted a cohort study of ICU patients dying in 10 medical centers in the Seattle-Tacoma area and found that family satisfaction with decision making was associated with the withdrawal of life support, and chart documentation of physician recommendations to withdraw life support.
TL;DR: Mortality can be well predicted using models that maximize reliance on objective pathophysiologic variables whereas minimizing input from billing data, and such models should be less susceptible to the vagaries of billing information and inexpensive to implement.
Abstract: Background:Clinically plausible risk-adjustment methods are needed to implement pay-for-performance protocols. Because billing data lacks clinical precision, may be gamed, and chart abstraction is costly, we sought to develop predictive models for mortality that maximally used automated laboratory d
TL;DR: The study method converted numeric clinical information to structured data with high accuracy, and enabled research and quality of care assessments for practices lacking structured data entry.