Hackathon: How Opus transforms emergency medical triage

What if you could conduct emergency medical triage from a smartphone?

Using Opus, the winner of the AI Genesis Hackathon hosted by LabLab.ai did just that.

In first place, resident physician and solo developer Fuyuto Miyake from Japan transformed a smartphone into a lightweight multimodal triage sensor and used Opus as the central decision engine for safe, auditable emergency triage.

Emergency triage today is slow, inconsistent, and high-risk, with individual judgement often leading to errors and subtle vital sign abnormalities often overlooked in the waiting room. Miyake notes that emergency room overcrowding also causes delayed evaluations and poorer patient outcomes.

Enter Opus. Miyake breaks down how he did it here:

A PWA (progressive web app) serves as a multimodal sensor, performing a 10-second on-device video analysis using MediaPipe Tasks and a lightweight rPPG pipeline (remote photoplethysmography, a technology that detects vital signs via camera) built with WebAssembly/FFT (Fast Fourier Transform, the mathematical algorithm used to process the data). This extracts respiratory rate, heart rate, posture, respiratory effort, pallor, sweat, eye openness, facial asymmetry, and pain indicators.

These structured features are assembled into a unified TriageRequestPayload and sent to an Opus workflow designed around the full Intake → Understand → Decide → Review → Deliver lifecycle.

In “Intake”, Opus validates the schema and receives vitals, posture, face features, and the chief complaint. In “Understand”, Opus’s Agent nodes detect vital-sign red flags, extract complaint keywords, and classify high-risk patterns such as severe respiratory effort or multiple unstable findings. Each Agent outputs structured fields with concise reasoning.

In “Decide”, a Decision node aggregates all extracted information to assign a triage level (red/yellow/green) using explicit rules, while an Agentic Review node checks guideline consistency and highlights mismatches. For “Review”, Opus routes only RED + high-risk cases to a “Human Review” node, creating a human-in-the-loop safety gate aligned with real emergency workflows. Reviewers receive extracted fields, triggered rules, and reasoning, and may confirm or modify the triage level with justification.

In “Deliver”, Opus generates a structured TriageResponsePayload containing the final triage level, a clinical summary, recommended next actions, and key debug fields. Each “Job” automatically produces an audit artifact capturing inputs, intermediate outputs, rule activations, timestamps, reviewer decisions, and model versions, enabling full provenance and compliance readiness.

This system shows how Opus can orchestrate a high-risk, high-volume medical workflow with transparency, safety, and practical deployability.

At AppliedAI, there is no room for hype – only real-world, measurable solutions. If an individual medical professional could build a transformational tool for such a vital process using Opus and a smartphone, what could Opus do for entire enterprises?

Find out for yourself. Get in touch to see what Opus can do to transform your business.