Related Work

Research & Publications

Explore related UTSW research and conference work from UT-REAL collaborators. UTSW studies are distinguished from consortium outputs. Summaries draw on the linked papers, preprints, and posters.

Published UTSW research

Peer-reviewed UTSW research describing the methods and deployments UT-REAL builds on. This includes work published before and after the multi-site award. These papers report UTSW studies rather than UT-REAL consortium outcomes.

Published Article in Press BMC Medical Education · November 13, 2026

How recording inputs affect multimodal AI agreement with reference scores in OSCE physical examinations: a retrospective ablation study

Shinyoung Kang, Michael J. Holcomb, Ameer H. Shakur, David Hein, Huong-Tra Ngo, Hunter Schuler, Philip C. Jarrett, Thomas O. Dalton, Andrew R. Jamieson

BMC Medical Education (2026). Published online November 13, 2026 (Article in Press).

Retrospective study of 300 OSCE encounters across six UTSW stations examining how recording inputs affect AI agreement with a local composite reference. Three-camera video with audio had the highest agreement (quadratic-weighted κ = 0.832; 95% CI 0.775–0.885), while silent video performed poorly. The AI–routine-evaluator difference was sensitive to one rubric item. The reference does not independently establish physical-examination competence; generalization beyond structured OSCEs remains untested.

UTSW retrospective study. Related research; not a UT-REAL consortium publication.

Published Article in Press npj Digital Medicine · October 7, 2026

Prospective deployment of multimodal AI grading for medical student OSCEs

Huong-Tra Ngo, Ameer Hamza Shakur, Michael Holcomb, Shinyoung Kang, David Hein, Judah Gruen, Hunter Schuler, Philip Jarrett, Thomas Dalton, Krystle Campbell, Daniel Scott, Andrew R. Jamieson

npj Digital Medicine (2026). Published online October 7, 2026 (Article in Press).

Prospective deployment of MAPLES at UTSW for 222 medical students across written notes, audio, and video, with human review of low-scoring assessments and physician adjudication of selected AI–human disagreements.

UTSW prospective Fall 2025 MAPLES deployment that provides a foundation for UT-REAL multi-site work. This paper reports the UTSW deployment, not UT-REAL consortium outcomes.

Published NEJM AI · November 27, 2024

Rubrics to Prompts: Assessing Medical Student Post-Encounter Notes with AI

Andrew R. Jamieson, Michael J. Holcomb, Thomas O. Dalton, Krystle K. Campbell, Sol Vedovato, Ameer Hamza Shakur, Shinyoung Kang, David Hein, Jack Lawson, Gaudenz Danuser, Daniel J. Scott

NEJM AI. 2024;1(12).

Peer-reviewed case study on using large language models to grade medical-student post-encounter notes from structured clinical rubrics.

Foundational UTSW research published before the UT-REAL multi-site award.

Published JMIR AI · December 18, 2025

Physical Examination Identification in Medical Education Videos: Zero-Shot Multimodal AI With Temporal Sequence Optimization Study

Shinyoung Kang, Michael Holcomb, David Hein, Ameer Hamza Shakur, Thomas Dalton, Andrew Jamieson

JMIR AI. 2025;4:e76586.

Zero-shot multimodal models were used to locate physical-examination segments in 500 fifteen-minute OSCE videos from five UTSW stations. At 1-second sampling, the Crossref abstract reports GPT-4o recall of 0.998 (95% CI 0.994–1.000) and intersection-over-union of 0.784 (95% CI 0.765–0.803).

Foundational UTSW research published before the UT-REAL multi-site award.

Published Discover Artificial Intelligence · August 8, 2025

Applying state-of-the-art artificial intelligence to grading in simulation-based education: assessment, feedback, and ROI

Krystle K. Campbell, Michael J. Holcomb, Sol Vedovato, Lenora Young, Gaudenz Danuser, Thomas O. Dalton, Andrew R. Jamieson, Daniel J. Scott

Discov Artif Intell. 2025;5:202.

Peer-reviewed paper on applying current AI methods to grading in simulation-based education, covering assessment, feedback, and return on investment.

Foundational UTSW research published before the UT-REAL multi-site award.

Accepted for publication

Related UTSW research accepted by a peer-reviewed journal and awaiting publication. Links lead to the available preprints. These are not UT-REAL consortium outputs.

None posted yet.

Preprints and manuscripts under review

Public preprints and manuscripts under journal review. They have not completed peer review. They are related UTSW research, not UT-REAL consortium outputs.

None posted yet.

Presentations

Conference posters and talks from the Jamieson Lab program.

Published Symposium on Artificial Intelligence for Learning Health Systems (SAIL) 2026, Puerto Rico · May 2026

Deploying a Multimodal AI Platform for Automated OSCE Grading in Medical Education

Huong-Tra Ngo

SAIL 2026 poster. Presented by Tra Ngo.

Poster presented by Tra Ngo describing MAPLES as a multimodal platform for OSCE grading across written notes, communication, and physical examination. A walkthrough is on this site.

Jamieson Lab presentation of MAPLES deployment work. Not a UT-REAL consortium output.

UT-REAL consortium outputs

Conference work from UT-REAL collaborators on governance and partner-site assessment practice. Each entry describes its specific scope.

Conference poster · Governance · October 2026

Continuous Governance of AI-Assisted Assessment in Medical Education: A Practice-Informed Framework

Presented by Huong-Tra Ngo

University of Utah · UT Southwestern Medical Center

A practice-informed framework for governing AI-assisted assessment throughout deployment, evaluation, and revision, with defined roles, human review, and readiness stages for collaboration across institutions.

The partner-readiness table shows status during poster preparation (September–October 2026) and is not updated here.

Authors

Huong-Tra Ngo, PhD; Ameer H. Shakur, PhD; Michael J. Holcomb, MS; Andrew Wong, MD, MS; Judah Gruen, MD; Andrew R. Jamieson, PhD.

2026 UT REAL Health AI Symposium · Posted October 27, 2026

Conference poster · Rubric design · October 2026

Ask a Better Question: How an AI Grader Sharpened Hypothesis-Driven History-Taking in a Medical Student Objective Structured Clinical Examination

Presented by Lindsey Stockton

UT Tyler School of Medicine · UT Southwestern Medical Center

A UT Tyler case study showing how discrepancies from an AI grader exposed ambiguous history-taking criteria and prompted more explicit, hypothesis-driven rubric questions.

A qualitative account of how one case’s rubric was revised.

Authors

Lindsey A Stockton, MD; Andrea Cooley, DO; Ameer Hamza Shakur, PhD; Andrew Jamieson, PhD; Judah Gruen, MD.

2026 UT REAL Health AI Symposium · Posted October 27, 2026

Conference poster · AI–human grading agreement · October 2026

Evaluating Agreement Between AI and Human Graders in Medical School Simulation

Presented by Ameer Hamza Shakur

UT Southwestern Medical Center · UT Tyler School of Medicine · UT Health San Antonio

An initial retrospective comparison of AI and human rubric-item ratings across six clinical cases at UT Tyler and UT Health San Antonio, with faculty review of disagreements to guide rubric refinement.

Subtitled “An Initial Look at a Multi-Institutional Study.” This original poster, supplied by the presenter, reflects the preliminary work shared in October 2026.

Authors

Judah Gruen, MD; Ameer H. Shakur, PhD; Lindsey Stockton, MD; John Cunningham, MD; Luyang Jin, MD; Michael J. Holcomb, MS; Andrew R. Jamieson, PhD.

2026 UT REAL Health AI Symposium · Posted November 13, 2026

Questions about citing this work?

For permissions, collaboration, or a walkthrough of the underlying platform, contact the team. Partner-school faculty should start on Participating in UT-REAL.