Gallery

See the Platform in Action

Conference posters, screenshots, architecture diagrams, and published figures from the UT-REAL collaboration, the MAPLES AI grading platform, and the OSCE simulation environments it serves.

Explore MAPLES with sample materials. Demo access is by invitation; choose “Request access” on the sign-in page to get started.

2026 UT REAL Symposium Posters

Explore grading agreement, governance, and rubric design through posters from the UT-REAL collaboration.

Presented October 7–8, 2026, in Austin. Read about the symposium.

Preview of the Continuous Governance poster by Ngo and colleagues

Conference poster · Governance · October 2026

Continuous Governance of AI-Assisted Assessment in Medical Education: A Practice-Informed Framework

Presented by Huong-Tra Ngo

University of Utah · UT Southwestern Medical Center

A practice-informed framework for governing AI-assisted assessment throughout deployment, evaluation, and revision, with defined roles, human review, and readiness stages for collaboration across institutions.

The partner-readiness table shows status during poster preparation (September–October 2026) and is not updated here.

Authors

Huong-Tra Ngo, PhD; Ameer H. Shakur, PhD; Michael J. Holcomb, MS; Andrew Wong, MD, MS; Judah Gruen, MD; Andrew R. Jamieson, PhD.

2026 UT REAL Health AI Symposium · Posted October 27, 2026

Preview of the Ask a Better Question poster by Stockton and colleagues

Conference poster · Rubric design · October 2026

Ask a Better Question: How an AI Grader Sharpened Hypothesis-Driven History-Taking in a Medical Student Objective Structured Clinical Examination

Presented by Lindsey Stockton

UT Tyler School of Medicine · UT Southwestern Medical Center

A UT Tyler case study showing how discrepancies from an AI grader exposed ambiguous history-taking criteria and prompted more explicit, hypothesis-driven rubric questions.

A qualitative account of how one case’s rubric was revised.

Authors

Lindsey A Stockton, MD; Andrea Cooley, DO; Ameer Hamza Shakur, PhD; Andrew Jamieson, PhD; Judah Gruen, MD.

2026 UT REAL Health AI Symposium · Posted October 27, 2026

Preview of the AI and Human Graders agreement poster by Gruen, Shakur, and colleagues

Conference poster · AI–human grading agreement · October 2026

Evaluating Agreement Between AI and Human Graders in Medical School Simulation

Presented by Ameer Hamza Shakur

UT Southwestern Medical Center · UT Tyler School of Medicine · UT Health San Antonio

An initial retrospective comparison of AI and human rubric-item ratings across six clinical cases at UT Tyler and UT Health San Antonio, with faculty review of disagreements to guide rubric refinement.

Subtitled “An Initial Look at a Multi-Institutional Study.” This original poster, supplied by the presenter, reflects the preliminary work shared in October 2026.

Authors

Judah Gruen, MD; Ameer H. Shakur, PhD; Lindsey Stockton, MD; John Cunningham, MD; Luyang Jin, MD; Michael J. Holcomb, MS; Andrew R. Jamieson, PhD.

2026 UT REAL Health AI Symposium · Posted November 13, 2026

MAPLES Platform

The AI grading interface used by faculty and simulation staff.

MAPLES Dashboard

At the time of this screenshot, the MAPLES dashboard showed 46 active rubrics, 93 student groups, and 182 grading runs in the UTSW production system.

Rubric Management

Rubric management view: a rubric with 64 prompts across audio, video, and note modalities, organized by OSCE station. Each item is mapped to a grading mode and scoring level.

Rubric Scoring Levels

Rubric level definitions: each item has scoring criteria (1-4) that both human raters and AI use, designed to support consistent interpretation across graders.

Grading Management

Grading configuration: select a student group, map rubrics to stations, enable optional features (localizer, mosaic videos), and launch AI grading with one click.

Video Review

Multimodal review: synchronized video playback with AI-generated transcript and item-level assessment. Faculty can accept the AI score or flag for manual review.

Post-Encounter Note Review

Post-encounter note grading: the student's clinical note alongside the AI assessment, rationale, and rubric scoring guide. Faculty review each item with full context.

Score Review & Edit

Batch review interface: 6,780 items across an entire OSCE cohort. Faculty can filter, accept AI scores, or drill into individual items for manual review.

Cost Estimates

A captured MAPLES run shows 622 encounters graded for $26.10 total ($0.04 per encounter) using Gemini 2.5 Pro, with a token-level cost breakdown for institutional budgeting.

End-to-End Platform Overview

How MAPLES fits in the full workflow: (1) OSCE encounter captured via multi-angle cameras, (2) Platform — SimRubrics for rubric design, Elephant for multimedia cataloging, MAPLES for AI grading orchestration, (3) Human review with faculty adjudication and grades reported to students.

OSCE Setup & Simulation Environment

The clinical simulation environments where encounters are captured.

OSCE Encounters

A UTSW OSCE workflow: medical students rotate through standardized-patient scenarios (e.g., Itchy Eyes, Leg Pain, Memory Problems), with encounters recorded from overhead cameras. MAPLES supports the resulting grading workflow.

Platform Architecture & Published Figures

From the NEJM AI publication and technical report — the rubrics-to-prompts pipeline.

NEJM AI Workflow

The rubrics-to-prompts workflow published in NEJM AI — translating clinical rubrics into AI grading specifications.

Production Deployment

Production deployment model at UTSW showing the human-in-the-loop review pipeline.

Platform Pipeline

End-to-end grading pipeline: data ingestion, AI assessment, quality review, and feedback delivery.

System Architecture

MAPLES platform architecture showing the relationship between SimRubrics, Elephant, and the grading engine.

Multimodal AI Assessment

Multi-camera video analysis using Gemini. These images are retained from the earlier preprint; the published BMC Medical Education Article in Press is linked alongside the original figure source.

Multimodal AI Framework

Multi-camera OSCE room setup with AI segmentation and grading pipeline — frontal, posterior, and overhead cameras feed into Gemini for physical examination assessment.

Multimodal Quadratic Weighted Kappa Results

Earlier preprint figure showing modality-ablation agreement in 300 OSCE encounters. See the BMC Medical Education Article in Press for the published estimates, local composite-reference methods, and the sensitivity of the AI–routine-evaluator comparison to one rubric item.

Item-Level F1 Heatmap

Macro F1 scores by rubric item and modality configuration — showing how different physical exam maneuvers benefit from different camera angles and input types.

Ready to explore MAPLES? Try the live demo with sample materials. Access is by invitation; choose “Request access” on the sign-in page.

For a start-to-finish walkthrough, read the MAPLES demo post, watch the MAPLES YouTube demo, or try the Wayfinder path: rubric authoring and encounter upload + grading. Have questions about the platform? Get in touch.