OASIS: Open Assessment and Scoring Infrastructure Stack

Rubric-based multimodal assessment using large language models

Project information, interactive guides, recorded product tours, and technical report for OASIS

An AI oasis in the desert of manual grading.

OASIS grades video, audio, and text with evaluator-defined rubrics and your choice of hosted or self-hosted language model. It plans each run around the material involved, records how every score was produced, and keeps people in control from rubric setup through final review.

See OASIS in action

Explore the CLI & TUI

See how the terminal checks that OASIS is ready, builds a workflow, follows a run, opens results, traces evidence, and browses data stored in Elephant. Everything shown is a sanitized recording, so this page never connects to a live service.

Open the CLI & TUI tour

Explore MAPLES

Follow three clearly labeled demo records through group setup, rubric review, station matching, run status, source-note inspection, and a human review that is deliberately left unsaved. There is no real learner data here: the scores are fixture values, and simply viewing the tour makes no service or model calls.

Open the MAPLES tour

Follow a complete CLI & MCP run

Watch OASIS check three synthetic Candle Making notes, validate the rubric, preview the work and estimated cost, try one note, run all three, reuse the results through MCP, and verify the evidence. The recording uses a predictable local test fixture, never calls an external model provider, and incurs $0 in provider charges.

Open the Candle Making tour

Download the overview GIF

Recorded terminal · silent

Load and check data in Elephant

Start with an agent-prepared manifest, let OASIS validate and preview it, pause for explicit approval, then open the imported group in the TUI and drill down through encounters and file counts. A final read-only CLI query confirms the stored file types, byte counts, and fingerprints.

Open the Elephant ingestion tour

Browse all guided tours

New interactive guide

Improve a rubric, one decision at a time

Watch an intentionally vague bicycle brake-adjustment rubric evolve from v1 to v3. Prepared Rubric Maker-style feedback leads to a human-selected v2; a synthetic dry run exposes one more scoring problem; and that feedback produces a focused v3 candidate. Every source and version stays visible. It is an illustrative click-through—not a recorded Rubric Maker or grading run.

Start the rubric improvement guide

Technical report

For readers who want the details, the report explains the architecture, how rubrics become model prompts, how each media type is graded, how the command-line and agent interfaces work, and how OASIS records evidence.

• Read and cite the preprint on arXiv
• Read the technical report online
• Download the technical report (PDF)

Related projects and demonstrations

The separate OASIS demo repository provides the public demo distribution entry point and frozen application-image references. Complete installation uses a separately supplied assisted kit; the demo repository states its current availability and terms.

MAPLES is the web application for grading and review. Wayfinder Rubric Studio helps authors build rubrics inside MAPLES, while Wayfinder Operator is the related—but separate—CLI/TUI experience.

• Explore UT-REAL, a multi-site MAPLES project
• Watch Ameer Hamza Shakur demonstrate the MAPLES grading and faculty-review workflow
• Watch Minhan Park and Licheng Yi demonstrate rubric authoring with Wayfinder in MAPLES

Publication scope

This publication includes project information, interactive guides, recorded demonstrations, and the technical report. Each experience explains what data it uses and whether anything was connected live. It does not yet include application source code, binaries, installation packages, sample datasets, or a tagged software release; any future release will state exactly what it contains and which terms apply.