🤖 AI

Human-in-the-Loop Multi-Agent Ventilator Decision Support with Contextual Bandit Preference Learning

This paper proposes the Ventilator Decision Support System (VDSS), a human-in-the-loop multi-agent framework that combines contract-driven modular coordination with contextual bandit preference learning to generate auditable, personalized ventilator recommendations that improve clinician acceptance and reduce interaction rounds.

Original authors: Sijia Li, Xiaoyu Tan, Qixing Wang, Weiyi Zhao, Chen Zhan, Teqi Hao, Xuemin Wang, Lei Gu, Roland Eils, Xihe Qiu

Published 2026-05-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sijia Li, Xiaoyu Tan, Qixing Wang, Weiyi Zhao, Chen Zhan, Teqi Hao, Xuemin Wang, Lei Gu, Roland Eils, Xihe Qiu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a patient in the Intensive Care Unit (ICU) on a ventilator. This machine is like a life-support puppet master, controlling how the patient breathes. But the patient's body is constantly changing—like a boat rocking on choppy waves. The doctors and nurses (the "clinicians") have to constantly tweak the machine's settings to keep the patient safe and comfortable.

Doing this is incredibly hard. It requires deep expertise, constant attention, and quick decisions. If they get it wrong, it's dangerous. If they get it right, the patient recovers.

The paper introduces a new tool called VDSS (Ventilator Decision Support System) to help these medical teams. Instead of being a "black box" that just spits out an answer, VDSS is designed as a team of specialized AI assistants working together under strict rules, with a human doctor always in the driver's seat.

Here is how it works, broken down into simple concepts:

1. The "Specialized Team" (Multi-Agent Framework)

Imagine you are building a complex house. You wouldn't ask one person to do the plumbing, the electrical work, the architecture, and the safety inspection all at once. You'd hire a team of experts.

VDSS does the same thing. It doesn't rely on one giant AI brain. Instead, it has a team of smaller AI "agents," each with a specific job:

The Waveform Analyzer: Looks at the squiggly lines on the monitor (like reading a heartbeat) to spot trouble spots.
The Detective: Checks the patient's vital signs to see what's wrong.
The Goal Setter: Decides what the team should try to achieve right now (e.g., "Let's get more oxygen" or "Let's calm the breathing down").
The Safety Inspector: A strict rule-follower that checks every suggestion against a list of "Do Not Touch" rules to ensure nothing dangerous is proposed.

These agents talk to each other through a structured "contract," ensuring they don't get confused or give contradictory advice.

2. The "Human-in-the-Loop" (The Pilot and the Co-Pilot)

This is the most important part. The AI is not the pilot; it is the co-pilot.

The AI team proposes a plan (e.g., "Let's turn the oxygen up a little").
They show this plan to the real doctor, along with a clear explanation of why they suggested it.
The doctor can say "Yes, do it" or "No, that's not right."

If the doctor says "No," the system doesn't just guess again randomly. It asks, "What specifically was wrong?" and then re-plans only the part that was rejected, saving time and energy. It's like a GPS that says, "You missed that turn? Okay, let's recalculate the route from where you are now," rather than sending you back to the start.

3. Learning Your "Style" (Contextual Bandit)

Every doctor has their own "fingerprint" or style. Some are very cautious and make tiny, slow changes. Others are bold and like to make bigger adjustments quickly. Some prioritize oxygen, while others prioritize comfort.

VDSS has a special learning engine called a Contextual Bandit. Think of this as a smart tutor that learns your teaching style.

Every time a doctor accepts a suggestion, the system takes a mental note: "Ah, Dr. Smith likes small, safe steps."
Every time a doctor rejects a suggestion, the system learns what not to do next time.
Over time, the system gets better at predicting exactly what that specific doctor would want to do, making the suggestions more useful and requiring fewer back-and-forth arguments.

4. The "Paper Trail" (Traceability)

In a hospital, you can't just trust a computer. You need to know why it made a decision.
VDSS keeps a detailed, step-by-step log of everything. It records:

What the patient looked like.
What the AI agents thought.
What the doctor decided.
Why the doctor accepted or rejected a plan.

This creates a clear "audit trail," like a flight recorder on a plane, so anyone can look back later and understand exactly how a decision was made.

What Did They Find?

The researchers tested this system using past patient records (like a flight simulator for doctors). They found that:

Better Accuracy: The AI team made fewer mistakes than a single AI model trying to do everything alone.
Faster Agreements: Because the system learned the doctors' styles and fixed only the rejected parts, doctors had to say "No" fewer times before agreeing on a plan.
Higher Trust: Doctors rated the suggestions as clearer, safer, and more useful.

The Bottom Line

The paper argues that in high-stakes situations like the ICU, you don't want a robot that takes over. You want a smart, organized team of assistants that helps the human expert think faster and safer, learns from their specific style, and keeps a perfect record of every step. It's about collaboration, not replacement.

Technical Summary: Human-in-the-Loop Multi-Agent Ventilator Decision Support with Contextual Bandit Preference Learning

1. Problem Statement

Mechanical ventilation in the Intensive Care Unit (ICU) requires continuous, fine-grained adjustments to match evolving patient physiology, gas exchange, and hemodynamics. Current clinical practice relies on protocols and clinician experience, yet faces significant challenges:

Dynamic Complexity: Patient trajectories are non-stationary, and strategies that work for one patient or time point may fail for another.
Workflow Constraints: Bedside titration demands rapid responses to imperfect signals, often leading to inconsistent adjustments due to time pressures and shift handoffs.
Personalization Gaps: Clinical teams vary in risk tolerance, tuning styles, and local conventions. Existing decision support systems (DSS) often fail to adapt to these specific preferences, limiting trust and adoption.
Limitations of Current AI: Rule-based approaches lack generalization for personalization, while end-to-end reinforcement learning (RL) or single Large Language Model (LLM) systems are difficult to control, audit, and align with specific safety boundaries.

The authors argue that in high-risk clinical settings, pointwise model performance is insufficient. Successful deployment requires a system where behavior is inspectable at the bedside, reasoning is traceable, and the system can adapt to clinician preferences without breaking the clinical workflow.

2. Methodology: The Ventilator Decision Support System (VDSS)

The authors propose VDSS, a human-in-the-loop (HITL) multi-agent framework designed to decompose ventilator decision-making into modular components regulated by contract-driven structured interfaces.

2.1 Multi-Agent Architecture

Rather than a single model mapping inputs to outputs, VDSS orchestrates a workflow of specialized agents (LLM or VLM modules) with defined roles and schema-constrained outputs:

Data Ingestion: A Waveform Analyzer converts pressure and flow traces into structured cues (e.g., asynchrony, uncertainty) to complement tabular data.
Detection & Goal Inference: A Detection Agent synthesizes cues into a state summary. A Phase Goal Manager infers the treatment phase and specifies primary/secondary goals.
Decision Gating: A Hold/Adjust Gate decides whether to maintain current settings or initiate titration, preventing unnecessary adjustments when evidence is insufficient.
Planning & Safety: If adjustment is needed, a Strategy Selector, Mode Select Agent, and Parameter Planner generate a compact set of executable updates. These proposals undergo deterministic safety and compatibility checks before reaching the clinician.
Feedback & Revision: If a clinician rejects a proposal, a Reflect Agent translates the feedback into revision constraints, identifying the minimal decision layer (e.g., strategy vs. parameters) to revisit. This enables targeted replanning rather than full pipeline reruns.
Documentation: A Note Generator produces a structured summary and an auditable cycle record upon acceptance.

2.2 Human-in-the-Loop Interaction

The system operates in adjustment cycles. For each cycle $t$ , the system proposes settings $a^{(k)}_t$ in rounds $k=1, \dots, K_t$ .

Rejection Handling: Rejection is treated as a structured control signal. The system records the rationale, localizes the error to a specific decision stage, and replans only that stage.
Termination: The cycle ends when the clinician accepts a proposal ( $\tilde{a}_t$ ), which becomes the ground truth for that cycle.

2.3 Online Preference Adaptation (Contextual Bandit)

To address the variability in clinician tuning styles, VDSS employs an online contextual bandit mechanism:

Preference State: Each clinician $d$ maintains a preference state $\theta_d$ .
Feedback Loop: Upon cycle closure, the system extracts a preference signal $p_t$ based on the final accepted decision $\tilde{a}_t$ and the interaction trace $h_t$ .
Learning Objective: The bandit updates $\theta_d$ using the formula $\theta_d \leftarrow \text{BanditUpdate}(\theta_d; x_t, \tilde{a}_t, h_t, p_t)$ .
Preference Categories: The model learns over 12 fixed categories, including granularity (small steps vs. assertive), mode changes, prioritization of clinical objectives (oxygenation, lung protection, hemodynamics, etc.), and workflow tendencies (e.g., deferring action when data is insufficient).
Integration: The updated $\theta_d$ serves as context for the reasoning engine in subsequent cycles, guiding candidate ranking and selection.

3. Key Contributions

Modular Multi-Agent Framework: VDSS introduces a contract-driven, multi-agent architecture that decomposes ventilator titration into verifiable, maintainable modules, replacing monolithic end-to-end models with a coherent, auditable reasoning trace.
Structured Rejection & Targeted Replanning: The system implements a mechanism where clinician rejection triggers targeted revision of specific decision layers, reducing unproductive iterations and maintaining interaction stability.
Online Preference Adaptation: The integration of a contextual bandit allows the system to learn individual clinician tuning styles from accepted decisions, improving recommendation alignment without requiring explicit preference elicitation.
Traceable Evidence: The framework ensures that every recommendation is accompanied by a structured summary, safety checks, and a rationale, facilitating clinical review and audit.

4. Experimental Results

The authors evaluated VDSS using retrospective ICU trajectory replay and expert review on a dataset of 1,309 structured records (7,447 setting entries) covering 13 ventilation modes.

Performance vs. Single Models: When deployed within VDSS, models significantly outperformed direct single-model generation. For example, using GPT-5.2 as the backbone:
- MSE (Mean Squared Error): Reduced from 0.343 (single model) to 0.102 (VDSS).
- Overall Clinician Rating: Improved from 2.63 to 4.11 (on a 1–5 scale).
- Acceptability: Increased from 2.75 to 4.09.
Ablation Studies: Removing waveform evidence (NoImg) or preference context (NoPref) resulted in degraded performance (e.g., MSE rose to 0.145 and 0.128 respectively), confirming the necessity of both dynamic evidence and preference learning.
Interaction Efficiency: Analysis of "regret" (number of rejected proposals per cycle) showed a downward trend over 100 cycles, indicating that the system learned clinician preferences and required fewer interaction rounds to reach an acceptable plan.
Runtime: On local hardware (RTX 5070 Ti), the average cycle time was 305.3 seconds, with waveform analysis being the primary bottleneck (56.46 s). The framework maintained a completion failure rate below 7%.

5. Significance and Claims

The paper claims that VDSS represents a practical step toward clinically deployable human-AI collaboration in high-risk settings. Its significance lies in:

Safety and Auditability: By enforcing contract-driven interfaces and deterministic safety checks, the system ensures that generative planning remains bounded by explicit constraints, addressing the "black box" concerns of end-to-end RL.
Workflow Alignment: The HITL design respects the iterative nature of bedside titration, allowing clinicians to override, reject, and refine suggestions without breaking the loop.
Personalization without Overfitting: The contextual bandit approach adapts to individual clinician styles using real-world feedback (accepted decisions) rather than requiring massive pre-training on specific user data, making it scalable across different institutions and practitioners.
Interpretability: The system provides a transparent link between the recommendation, patient state, and team preferences, supporting efficient repeated use and trust.

The authors conclude that while pointwise model accuracy is necessary, the framework's ability to provide verifiable, preference-aware, and safe collaboration is the critical factor for sustained adoption in critical care.