Peer-reviewed UTSW research describing the methods and deployments UT-REAL builds on. This includes work published before and after the multi-site award. These papers report UTSW studies rather than UT-REAL consortium outcomes.
Published Article in Press BMC Medical Education · November 13, 2026
How recording inputs affect multimodal AI agreement with reference scores in OSCE physical examinations: a retrospective ablation study
Shinyoung Kang, Michael J. Holcomb, Ameer H. Shakur, David Hein, Huong-Tra Ngo, Hunter Schuler, Philip C. Jarrett, Thomas O. Dalton, Andrew R. Jamieson
BMC Medical Education (2026). Published online November 13, 2026 (Article in Press).
Retrospective study of 300 OSCE encounters across six UTSW stations examining how recording inputs affect AI agreement with a local composite reference. Three-camera video with audio had the highest agreement (quadratic-weighted κ = 0.832; 95% CI 0.775–0.885), while silent video performed poorly. The AI–routine-evaluator difference was sensitive to one rubric item. The reference does not independently establish physical-examination competence; generalization beyond structured OSCEs remains untested.
UTSW retrospective study. Related research; not a UT-REAL consortium publication.
Published Article in Press npj Digital Medicine · October 7, 2026
Prospective deployment of multimodal AI grading for medical student OSCEs
Huong-Tra Ngo, Ameer Hamza Shakur, Michael Holcomb, Shinyoung Kang, David Hein, Judah Gruen, Hunter Schuler, Philip Jarrett, Thomas Dalton, Krystle Campbell, Daniel Scott, Andrew R. Jamieson
npj Digital Medicine (2026). Published online October 7, 2026 (Article in Press).
Prospective deployment of MAPLES at UTSW for 222 medical students across written notes, audio, and video, with human review of low-scoring assessments and physician adjudication of selected AI–human disagreements.
UTSW prospective Fall 2025 MAPLES deployment that provides a foundation for UT-REAL multi-site work. This paper reports the UTSW deployment, not UT-REAL consortium outcomes.
Published NEJM AI · November 27, 2024
Rubrics to Prompts: Assessing Medical Student Post-Encounter Notes with AI
Andrew R. Jamieson, Michael J. Holcomb, Thomas O. Dalton, Krystle K. Campbell, Sol Vedovato, Ameer Hamza Shakur, Shinyoung Kang, David Hein, Jack Lawson, Gaudenz Danuser, Daniel J. Scott
NEJM AI. 2024;1(12).
Peer-reviewed case study on using large language models to grade medical-student post-encounter notes from structured clinical rubrics.
Foundational UTSW research published before the UT-REAL multi-site award.
Published JMIR AI · December 18, 2025
Physical Examination Identification in Medical Education Videos: Zero-Shot Multimodal AI With Temporal Sequence Optimization Study
Shinyoung Kang, Michael Holcomb, David Hein, Ameer Hamza Shakur, Thomas Dalton, Andrew Jamieson
JMIR AI. 2025;4:e76586.
Zero-shot multimodal models were used to locate physical-examination segments in 500 fifteen-minute OSCE videos from five UTSW stations. At 1-second sampling, the Crossref abstract reports GPT-4o recall of 0.998 (95% CI 0.994–1.000) and intersection-over-union of 0.784 (95% CI 0.765–0.803).
Foundational UTSW research published before the UT-REAL multi-site award.
Published Discover Artificial Intelligence · August 8, 2025
Applying state-of-the-art artificial intelligence to grading in simulation-based education: assessment, feedback, and ROI
Krystle K. Campbell, Michael J. Holcomb, Sol Vedovato, Lenora Young, Gaudenz Danuser, Thomas O. Dalton, Andrew R. Jamieson, Daniel J. Scott
Discov Artif Intell. 2025;5:202.
Peer-reviewed paper on applying current AI methods to grading in simulation-based education, covering assessment, feedback, and return on investment.
Foundational UTSW research published before the UT-REAL multi-site award.