💻 computer science

A Multi-Agent Clinical Decision Support System with Patient Similarity Retrieval on MIMIC-IV

This paper presents a multi-agent clinical decision support system that leverages MIMIC-IV data and large language models to perform real-time patient similarity retrieval and generate temporally-aware clinical analyses through a six-agent orchestration workflow.

Original authors: Germano Ammirata, Salvatore Contino, Luca Cruciata, Emanuele Principato, Camillo Gioè, Roberto Pirrone

Published 2026-09-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Germano Ammirata, Salvatore Contino, Luca Cruciata, Emanuele Principato, Camillo Gioè, Roberto Pirrone

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Hospitals generate a staggering amount of data every day. From the moment a patient walks through the door, a digital record begins to form, capturing everything from their age and vital signs to the specific medications they receive and the complex web of diseases they carry. For decades, doctors have relied on this information to make life-or-death decisions, but the sheer volume of records has often made it difficult to find the specific patterns hidden within. Traditional computer tools could crunch numbers or follow rigid rules, but they struggled to understand the nuance of a human question or to connect a current patient's story with the thousands of similar stories that came before. The challenge has been to build a system that can not only read these massive archives but also reason through them, offering a doctor a clear, evidence-based path forward when facing a complex case.

A team of researchers has now built a new kind of computer assistant designed to solve this problem. Instead of relying on a single, massive artificial intelligence program to do all the thinking, they created a team of six specialized digital workers, each with a distinct job. This system, tested on a vast collection of de-identified patient records from a major medical center, acts like a collaborative research unit. When a doctor asks a question about a patient, the system breaks the task down, sends parts of it to different specialists, and then weaves their findings into a single, coherent report. The goal was to move beyond simple predictions and provide transparent, contextual reasoning that a physician could trust and understand.

The process begins when a doctor types a question into the system, such as asking how a patient's condition compares to others or what the best treatment might be. The first digital worker, acting as a manager, reads the question and decides exactly what steps are needed. It determines whether the system should look for similar patients in the database, which specific details about the current patient are most important, and how to weigh different factors like age, lab results, or specific diagnoses. This manager then directs the other workers to gather the necessary information.

To find the right comparisons, the system uses a sophisticated method to understand what makes one patient similar to another. It does not just look for exact matches; instead, it translates the patient's medical history into a unique digital fingerprint. This fingerprint is built from three different angles: the patient's physical measurements like blood pressure and heart rate, their demographic details like age and sex, and their list of diagnoses. The researchers developed a special way to handle the list of diseases, treating them like a language where the order and combination of words matter. By analyzing how often certain conditions appear together across hundreds of thousands of records, the system learns that some diseases are closely related, even if they have different names. This allows it to find patients who share the same complex medical story, not just the same single symptom.

Once the manager has identified the most relevant similar patients, a second worker gathers all the raw data for both the current patient and the matched group. This data is then organized into a clear, structured format that is easy for the next worker to read. This step is crucial because it translates the messy, technical codes used in hospital databases into plain language, ensuring that the reasoning engine understands exactly what it is looking at.

The core of the system then takes over. A specialized reasoning engine analyzes the current patient alongside the group of similar cases. It looks for patterns, such as whether a patient's lab results are trending in a dangerous direction compared to others with similar histories, or whether a specific medication worked well for the matched group. This worker does not just guess; it grounds its conclusions in the actual data it has retrieved. If there are multiple aspects to the doctor's question, this engine can run several lines of reasoning at the same time, ensuring that no part of the puzzle is missed.

Finally, a synthesizer brings all these separate threads together. It takes the detailed analyses from the reasoning engine and writes a single, unified report. This final step is designed to eliminate contradictions and ensure the answer directly addresses the doctor's original question. The result is a narrative that explains not just what the data says, but why it matters for this specific patient, offering insights that are both safe and tailored to the individual's unique situation.

The researchers tested this system on a large set of real-world questions and found that it performed with high reliability. The manager worker correctly identified the right steps to take in nearly every case. The system was also highly successful at finding truly similar patients, often retrieving groups where the medical histories overlapped significantly with the target patient. Most importantly, the final reports generated by the system were medically sound and free of internal contradictions. The team demonstrated that by breaking a complex task into smaller, specialized steps, they could create a tool that is more accurate and trustworthy than a single, all-purpose artificial intelligence. This approach suggests a future where doctors are supported by a team of digital experts, each handling a specific part of the puzzle, to provide clearer and more reliable guidance for patient care.

Technical Summary: A Multi-Agent Clinical Decision Support System with Patient Similarity Retrieval on MIMIC-IV

Problem Statement

Current Clinical Decision Support Systems (CDSS) often rely on rigid rule-based tools or opaque machine learning classifiers that fail to provide transparent, contextualized reasoning tailored to specific physician inquiries. While Large Language Models (LLMs) offer advanced medical reasoning capabilities, deploying them as monolithic assistants presents critical bottlenecks: they lack the intrinsic ability to execute precise, large-scale retrieval of structured patient data, are prone to hallucinations, and struggle with the temporal interdependence of electronic health record (EHR) data. Existing Retrieval-Augmented Generation (RAG) approaches often focus on unstructured text rather than the complex, structured nature of clinical tables and ICD codes.

Methodology

The authors propose a multi-agent framework that integrates structured EHR data from MIMIC-IV v3.1 with LLM-driven reasoning. The system is built upon a six-agent architecture orchestrated through a structured pipeline, designed to decouple deterministic tasks from open-ended clinical synthesis.

1. Data Preprocessing and Embedding Construction

The system processes MIMIC-IV v3.1 data (364,627 patients, 546,028 admissions) into three semantically distinct embedding spaces to capture complementary patient dimensions:

Clinical Embedding: A 52-dimensional vector derived from physiological descriptors (vital signs, lab values, temporal slopes) normalized via standard scaling.
Categorical Embedding: An 8-dimensional one-hot encoded vector representing sex and age groups (six adult cohorts).
Diagnosis Embedding: The most innovative component, utilizing IDF-weighted FastText.
- ICD-9 codes are converted to ICD-10 and truncated to the fourth character to define disease categories.
- A FastText model (Skip-gram, 100 dimensions) is trained on these categories, treating admissions as "sentences."
- To handle the long-tail distribution of diagnoses, an Inverse Document Frequency (IDF) weighting scheme is applied with a power factor ( $\alpha=2.0$ ) to amplify rare, clinically relevant conditions while suppressing ubiquitous codes.
- The final patient embedding is the weighted average of the FastText vectors for their unique ICD categories.

2. Vector Similarity and Retrieval

Patient embeddings are indexed in a Milvus vector database using the IVF FLAT scheme. Retrieval occurs in two stages:

Stage 1 (ANN Candidate Generation): An Approximate Nearest Neighbor (ANN) search on the diagnosis embedding retrieves a large pool of candidates (e.g., $M=200$ ) to ensure high recall.
Stage 2 (Multi-Embedding Re-Ranking): Candidates are re-scored using exact cosine similarity across all three embedding types (diagnostic, clinical, categorical). The weights for this scoring ( $w_{diag}, w_{cli}, w_{cat}$ ) are dynamically assigned by the Orchestrator agent based on the specific clinical query (e.g., upweighting diagnosis for comorbidity queries, or clinical parameters for acute physiological queries).

3. Six-Agent Architecture

The workflow is orchestrated by six specialized agents:

Orchestrator: Converts free-text physician queries into a structured routing JSON. It determines the analysis topics, whether to invoke patient similarity retrieval, the number of similar patients ( $k$ ), and the dynamic weighting for the similarity engine.
Context Constructor: Coordinates data retrieval, compiles the target patient's data and the retrieved similar cohort, and formats the output into a unified structured JSON document including a human-readable ICD glossary.
Data Gatherer: Retrieves raw data and translates database identifiers into standardized, human-readable clinical labels to ground the LLM prompts.
Similarity Engine: Executes the two-stage retrieval pipeline against the Milvus database.
Specialist LLM: Performs domain-specific clinical reasoning (e.g., pattern analysis, pharmacological consultation) using a frontier model. It operates with temporal awareness and uses the retrieved cohort as context rather than relying solely on parametric knowledge.
Report Synthesizer: Consolidates parallel analyses from Specialist LLMs into a unified, coherent clinical narrative, strictly filtering for the physician's original query to minimize redundancy and hallucinations.

Key Results

The system was evaluated on a benchmark of 100 diverse clinical queries using a hybrid evaluation approach, including an "LLM-as-a-judge" for clinical metrics.

Orchestrator Accuracy: Achieved 0.960, demonstrating high reliability in routing decisions (selecting agents, activating retrieval, and selecting context attributes).
Similarity Engine Performance:
- Hit-Rate @ K: 0.886 (at Jaccard $\ge$ 0.2), indicating the system successfully finds viable comparable cases in the vast majority of queries.
- nDCG @ K: 0.875, confirming high-quality ranking of retrieved patients.
- Precision @ K: 0.491, showing a significant density of relevant matches.
Clinical Reasoning Quality:
- Clinical Validity: 0.876, indicating robust medical correctness and safety.
- Contradiction Rate: 0.000, demonstrating that the Report Synthesizer flawlessly consolidated parallel analyses without introducing internal contradictions or hallucinations.
UMAP Validation: Visualizations of the IDF-weighted FastText embedding space showed that patients with matching diagnostic codes formed dense, localized clusters, while background patients formed sparse outer structures, validating the topological accuracy of the embedding space.

Significance and Claims

The paper claims that this framework effectively addresses the limitations of monolithic LLMs in clinical settings by:

Decomposing Cognitive Load: Separating deterministic routing and retrieval from open-ended reasoning reduces error cascades and minimizes hallucination risks.
Enhancing Context Precision: The use of a dedicated Context Constructor and structured JSON assembly optimizes token usage and ensures the LLM is grounded in verified, patient-specific evidence rather than raw database formats.
Improving Retrieval Relevance: The IDF-weighted FastText embedding successfully captures the morphological hierarchies of ICD codes and the clinical value of rare conditions, enabling the retrieval of highly aligned patient cohorts.
Providing Transparent Reasoning: The multi-agent approach offers a modular, inspectable pipeline where routing logic and data sources are explicit, contrasting with the "black box" nature of single-model outputs.

The authors conclude that combining agentic decomposition with multimodal RAG provides a robust framework for integrating AI into high-risk clinical contexts, offering safer, more interpretable, and tailored insights. They note limitations regarding the single-institution data scope (MIMIC-IV) and the reliance on automated evaluation rather than human expert panels, suggesting future work in clinician-in-the-loop validation and real-time EHR integration.