An agentic multidisciplinary framework for diabetic retinopathy treatment planning

Technical Summary: An Agentic Multidisciplinary Framework for Diabetic Retinopathy Treatment Planning

Problem Statement
Diabetic retinopathy (DR) is a leading cause of vision loss, yet effective treatment planning is hindered by the necessity for multidisciplinary collaboration between ophthalmology and endocrinology. Current clinical workflows face significant barriers, including limited specialist resources in primary care settings, a heavy documentation burden (accounting for ~38% of the total time spent per patient encounter), and the difficulty of convening multidisciplinary team (MDT) consultations. While Large Language Models (LLMs) have shown promise in medical knowledge tasks, they struggle with complex, open-ended treatment planning. Existing AI approaches often focus on diagnostic labels or simplify treatment outputs into probability scores, failing to generate comprehensive, individualized plans. Furthermore, the lack of high-quality, structured treatment-planning datasets and robust evaluation frameworks limits the development and validation of AI systems in this domain.

Methodology
The authors propose a three-component framework to address these challenges:

1. DR Treatment-Planning Dataset:
   The study constructs a large-scale, publicly available dataset (~10,000 cases) linking fundus images, structured clinical records, and standardized treatment plans. The dataset comprises:

   • 100 Real-World Cases: Multi-center clinical data from Zhongshan Ophthalmic Center and Gaoyao District People's Hospital, covering mild to proliferative DR.
   • 1,040 Hybrid Real-Synthetic Cases: Generated from real fundus images using an "image-to-report-to-plan" pipeline, where initial AI-generated records and plans were rigorously reviewed and revised by senior ophthalmologists to meet clinical standards.
   • 8,908 Large-Scale Synthetic Cases: Automatically generated using the proposed agent system for future model training and fine-tuning.

2. DRAgent (Role-Based Multi-Agent System):
   DRAgent is a multi-agent system built on the DIFY platform, simulating a real-world MDT process. It employs distinct clinical personas:

   • Roles: An outpatient physician (data intake), an ophthalmologist (lead attending), a laser surgeon, an endocrinologist, a pharmacist, and a nurse (plan consolidation).
   • Workflow: The system processes fundus images and auxiliary reports to generate clinical records. The ophthalmologist drafts an initial plan, which is then iteratively discussed with the laser surgeon (for procedural details), endocrinologist (for metabolic management), and pharmacist (for medication safety). The nurse agent consolidates feedback into a structured final plan.
   • Knowledge Integration: Agents utilize Retrieval-Augmented Generation (RAG) with a summarization-based retrieval strategy, grounded in five authoritative Chinese clinical guidelines (e.g., DR diagnosis/treatment, laser guidelines, diabetes management, and essential medicines).

3. Multidimensional Evaluation Framework:
   To overcome the limitations of standard QA benchmarks, the authors introduce a three-pronged evaluation strategy:

   • Guideline Compliance: Measured via Normalized Compliance Rate (NCR) against specific dimensions (normativity, correctness, surgical requirements, decision points) for both ophthalmic and endocrine components.
   • Objective Metrics: Lexical and semantic similarity (BLEU, ROUGE-L, METEOR, BERTScore) calculated against physician-revised reference plans, utilizing weighted averaging to prioritize clinically critical sections over generic text.
   • Human Evaluation: A blinded dual-review protocol by senior ophthalmologists using a 5-point clinical rationality scale, with third-expert adjudication for disagreements.

Key Results

• Performance Superiority: Across four different backbone LLMs (GPT-5, Gemini 2.5 Flash, DeepSeek-V3.1, Qwen3.5-flash), DRAgent consistently outperformed the standalone models.
  • Guideline Compliance: DRAgent significantly improved NCR scores. For example, on hybrid data, endocrine NCR improved from ~35% (standalone) to ~63% (DRAgent) for GPT-5. Similar gains were observed in real clinical data from two independent centers.
  • Objective Similarity: DRAgent achieved higher BLEU, ROUGE, and BERTScore values, indicating better alignment with expert-revised plans in both terminology and clinical semantics.
  • Human Evaluation: The mean clinical rationality score increased from 4.02 (standalone LLMs) to 4.80/5.0 for DRAgent. This improvement was statistically significant across all backbone models.
• Ablation Studies: Removing specific agents degraded performance in their respective domains. Removing the laser surgeon reduced ophthalmic guideline adherence, while removing the endocrinologist and pharmacist significantly lowered endocrine management scores, confirming the complementary nature of the roles.
• Robustness: DRAgent demonstrated greater stability across different DR severity grades and backbone models compared to standalone LLMs, which showed higher sensitivity to model selection and disease severity.

Significance and Claims
The paper claims that DRAgent provides a viable starting point for developing AI systems for diseases requiring cross-specialty care. By transforming multidisciplinary decision-making into an evaluable, agent-based system, the framework addresses the "practice gap" where LLMs fail to translate knowledge into complex, individualized treatment plans. The authors emphasize that the performance gains are driven by the structured, role-based collaboration mechanism rather than the intrinsic capabilities of the underlying LLMs alone.

The study highlights the necessity of moving beyond diagnostic accuracy toward comprehensive treatment planning, supported by a new dataset and a multidimensional evaluation framework that balances guideline adherence, objective similarity, and human clinical judgment. The authors modestly note limitations, including the reliance on Chinese clinical guidelines (requiring validation for other healthcare systems), the current use of fundus images as the sole modality, and the need for future endocrinologist review in human evaluations. They position this work as a foundation for future research into adaptive agent collaboration and the application of similar frameworks to other chronic diseases.