DR. INFO at the Point of Care: A Prospective Pilot Study of an Agentic AI Clinical Assistant

This prospective pilot study demonstrates that DR. INFO, an agentic AI clinical assistant, significantly improves time efficiency and clinical decision support for physicians in Portuguese healthcare settings, achieving high satisfaction scores and a Net Promoter Score of 81.2 without evidence of attrition bias.

The Gist

Imagine a doctor's day is like trying to cook a five-course meal while simultaneously answering 50 phone calls, filling out tax forms, and remembering every single recipe ever written. That's the reality of modern medicine. Doctors spend more than half their time wrestling with paperwork and searching for information, leaving them exhausted and burnt out.

Enter DR. INFO, a new "digital sous-chef" designed to help. This isn't just a chatbot that guesses answers; it's an agentic AI assistant. Think of it as a super-smart research librarian who doesn't just guess the answer but goes straight to the library's most trusted, peer-reviewed books to find the facts before speaking.

Here is a simple breakdown of the study testing this tool:

🍳 The Problem: The Overwhelmed Chef

Doctors are drowning in data. They need to know the latest treatments, drug dosages, and diagnosis rules instantly, but the sheer volume of medical literature is too big for any human brain to hold. This leads to stress and mistakes.

🤖 The Solution: The Smart Assistant

The researchers tested a tool called DR. INFO. Unlike standard AI that might "hallucinate" (make up facts that sound real but are wrong), DR. INFO is built to check its sources first. It's like having a co-pilot who double-checks the map before you drive.

🧪 The Experiment: A Two-Week Trial

The team invited 29 doctors and medical students in Portugal to use DR. INFO for five days over two weeks.

• The Setup: They used the tool during their normal workdays, asking it things like "What's the best treatment for this condition?" or "How much of this drug should I give?"
• The Goal: To see if it saved them time and helped them make better decisions.

📊 The Results: A Standing Ovation

The feedback was overwhelmingly positive, like a restaurant getting a 5-star review from food critics who are used to great food.

1. Time Saver: Doctors felt the tool saved them a significant amount of time (rated 4.3 out of 5). It was like having a turbo button for information retrieval.
2. Decision Support: It helped them feel more confident in their choices (rated 4.2 out of 5).
3. The "Would You Recommend It?" Score: The Net Promoter Score (NPS) was 81.2. In the world of customer satisfaction, a score above 50 is excellent; 81 is "world-class." No one said they disliked it.

👥 Who Liked It Best?

• The Students & Residents (The Trainees): They loved it the most. It was like having a wise mentor available 24/7 to explain complex cases or check their study notes.
• The Veterans (Senior Doctors): Even the experienced doctors liked it, though they used it differently. Instead of learning new things, they used it as a "sanity check" to quickly verify drug dosages or confirm their instincts against the latest research.
• The Boss (Head of Service): One senior leader gave it a lower score for "decision support" but still liked the time savings. This makes sense; if you've been a chef for 30 years, you don't need a recipe book to tell you how to chop an onion, but you still appreciate a tool that brings you the ingredients faster.

⚠️ The Only Hiccups

It wasn't perfect. The main complaints were:

• Speed: Sometimes the tool took a little too long to "think" and answer because it was doing deep research. (Imagine a librarian running to the back of the library to find a book; it takes a moment).
• Specificity: Sometimes the answers were a bit too general, like a GPS giving directions to the whole city instead of the specific street.

🏁 The Bottom Line

This study is like a successful "beta test" for a new app. It showed that when you give doctors a smart, fact-checking AI assistant, they love it. It doesn't replace the doctor; it just clears the clutter off their desk so they can focus on the patient.

The Takeaway: Doctors are ready for AI, but only if it's reliable, fast, and actually helps them get back to what they do best: caring for people. The next step is to test this on a much larger scale to see if it truly reduces burnout and improves patient health.

Technical Summary

1. Problem Statement

The medical field faces a critical bottleneck where the volume of biomedical literature and clinical data exceeds the capacity of individual clinicians to assimilate it. This gap contributes to:

• Administrative Burden: Physicians spend only ~47% of their workweek in direct patient care, with the remainder consumed by Electronic Health Record (EHR) documentation and administrative tasks.
• Burnout: This administrative load is a primary driver of physician burnout (reported at 43.2% nationally in the US).
• AI Adoption Barriers: While Large Language Models (LLMs) show promise, their clinical adoption is hindered by safety concerns, specifically hallucinations (generating plausible but factually incorrect information) and a lack of source reliability in standalone models.

2. Methodology

The study was a prospective, single-arm pilot study conducted in October 2025 across Portuguese healthcare institutions.

• Intervention: DR. INFO v1.0, an "agentic" AI clinical assistant. Unlike standalone LLMs, DR. INFO uses an architecture that retrieves information from curated medical knowledge bases and peer-reviewed literature before generating a response, aiming to mitigate hallucinations.
• Participants: 29 clinicians (25 physicians, 4 medical students) across various specialties (Family Medicine, Intensive Care, Internal Medicine, etc.) and career stages.
• Protocol:
  • Participants integrated DR. INFO into their daily workflow for 5 working days over a two-week period.
  • Data Collection:
    • Daily eCRFs (eCRF 2): Likert-scale ratings (1–5) for "Time Saving" and "Decision Support."
    • Baseline eCRF (eCRF 1): Demographics, years of practice, and technology comfort.
    • Final eCRF (eCRF 3): Net Promoter Score (NPS) and open-ended qualitative feedback.
• Statistical Analysis: Due to the small sample size and ordinal data, non-parametric methods were used (Mann-Whitney U, Friedman test, Spearman rank correlation, Kruskal-Wallis H). Confidence intervals were calculated via percentile bootstrapping.

3. Key Contributions

• Agentic Architecture Validation: The study provides real-world evidence that coupling LLMs with retrieval-augmented generation (RAG) from curated medical sources improves clinician trust and utility compared to standalone models.
• Benchmarking Context: The paper references a companion technical study (HealthBench) where DR. INFO significantly outperformed standalone models (GPT-5, Grok 3, Gemini) and other clinical AI assistants (OpenEvidence, DoxGPT) on "Hard" clinical subsets (Score: 0.68 vs. 0.40–0.49 for others).
• Real-World Usability: Unlike previous studies focusing on isolated tasks, this pilot assessed the tool within routine, high-acuity clinical workflows across diverse specialties.
• Bias Mitigation Analysis: The authors conducted rigorous attrition analysis and sensitivity testing on the NPS to address potential survivorship bias, given that only 55% of participants completed the final survey.

4. Key Results

A. Perceived Utility

• Time Saving: Mean score of 4.27/5 (95% CI: 3.97–4.57). 87.8% of diary entries indicated agreement or strong agreement.
• Decision Support: Mean score of 4.16/5 (95% CI: 3.86–4.45). 85.6% agreed it improved diagnostic or therapeutic reasoning.
• Stability: Scores remained stable across all five study days (Friedman test: $p > 0.05$), indicating sustained utility rather than a "novelty effect."

B. Satisfaction (NPS)

• Observed NPS: 81.2 (based on 16 respondents), with 81% Promoters, 19% Passives, and 0% Detractors.
• Sensitivity Analysis: Even under conservative assumptions where all 13 non-respondents were classified as "Passives," the NPS remained positive at 44.8. If all were "Detractors," the NPS would be 0.0. This suggests the high satisfaction is robust.

C. Subgroup and Correlation Findings

• Career Stage: Junior clinicians (Students/Residents) reported the highest perceived value for decision support (Mean ~4.7), while senior clinicians (Attending/Consultants) used the tool primarily for verification and drug dosing.
• Experience Correlation: A trend was observed where years of practice inversely correlated with perceived decision support ($\rho = -0.381, p = 0.055$), suggesting the tool is particularly valuable for medical education and less experienced clinicians.
• Technology Comfort: No correlation was found between a user's self-reported tech comfort and perceived time savings, indicating the interface is accessible regardless of technical proficiency.

D. Qualitative Feedback

• Strengths: Speed of information retrieval, source transparency, and utility in complex cases.
• Weaknesses: The primary concerns were response latency (50% of respondents) and answer specificity (38%), particularly for nuanced clinical queries.

5. Significance and Limitations

Significance:

• Workflow Integration: The study demonstrates that AI can be successfully integrated into existing clinical workflows without disrupting established routines, even for users already familiar with traditional decision support tools (e.g., UpToDate).
• Burnout Mitigation: By addressing the "time saving" metric, the tool offers a potential pathway to reduce the cognitive load and administrative burden contributing to physician burnout.
• Regulatory Alignment: The design adheres to the EU AI Act principles of transparency and human oversight, making it a model for compliant high-risk AI in healthcare.

Limitations:

• Subjectivity: Outcomes were based on self-reported perceptions rather than objective clinical accuracy audits or patient outcome measures.
• Sample Size: The pilot nature (N=29) limits statistical power, particularly for subgroup analyses and detecting temporal trends.
• Ceiling Effects: High baseline satisfaction with existing tools may have limited the ability to distinguish degrees of improvement.
• Generalizability: The study was conducted exclusively in Portugal; cultural and systemic differences in other healthcare environments may affect adoption.

Conclusion:
DR. INFO represents a promising step toward safe, agentic AI in clinical practice. It achieved high clinician satisfaction and perceived efficiency gains in a real-world setting. Future research requires larger, controlled trials with objective performance metrics to validate whether these perceived benefits translate into improved patient outcomes and reduced burnout.