Technical Summary: ClinQueryAgent

Problem Statement

Healthcare organizations possess vast quantities of patient data, yet interrogating this data to identify specific patient cohorts for population health management remains a complex, labor-intensive task. Clinicians typically rely on specialist data analysts to translate natural language questions into sophisticated database queries. While existing text-to-query systems exist, they often struggle with the semantic complexity of medical terminology (polysemy, synonyms, abbreviations), the "context rot" associated with long multi-turn dialogues, and the critical requirement of maintaining patient data privacy when utilizing powerful cloud-based Large Language Models (LLMs).

Methodology

The authors introduce ClinQueryAgent, a task-oriented conversational agent designed to translate natural language population health questions into executable database queries. The system operates within the eHealthScope platform used by NHS Nottingham and Nottinghamshire.

Architecture and Privacy

The system employs a structural privacy design where the LLM agent is strictly isolated from the raw patient data (the execution layer). The agent operates on a reasoning layer comprising medical concepts and query logic. It interacts with the database only through specific tools that generate executable SQL queries, which are then run by the host system. This ensures no patient data leaves the secure NHS environment, even while leveraging frontier generative models deployed in the cloud.

Multi-Agent Delegation

To address the degradation of accuracy and context bloat ("context rot") in multi-turn conversations, the authors implement a sub-agent delegation mechanism:

• Query Agent: The primary agent responsible for understanding user intent, constructing logical query structures, and generating the final SQL.
• Retrieval Agent: A subordinate agent instantiated on-demand to handle the search for specific medical concepts within the local database and external knowledge bases.
• Workflow: When a user requests a concept (e.g., "Ozempic"), the Query Agent delegates the search to the Retrieval Agent. The Retrieval Agent searches the local database and the Unified Medical Language System (UMLS) to resolve synonyms and brand names (e.g., mapping "Ozempic" to "Semaglutide"). It returns a concise, relevant shortlist of concepts with explanations, allowing the Query Agent to maintain a focused context window.

Tooling and Knowledge

The system utilizes three primary tools:

1. Lexical Concept Search: Fetches concepts containing specific strings from the local NHS database.
2. Cohort Creation: Converts agent-generated logical queries into executable SQL and caches the resulting patient cohort.
3. URL Generation: Provides a link for the user to view the results in the eHealthScope UI.
   Additionally, the system integrates UMLS and OHDSI vocabularies to assist in mapping brand names to generic drug names and handling complex medical terminology.

Key Contributions

1. Real-World Evaluation: The system was evaluated in a live population health management platform (eHealthScope) used by 128 staff across 15 healthcare practices, covering 148,319 patients.
2. Privacy-Preserving Design: Demonstrated a method to securely deploy cloud-based generative models in sensitive healthcare environments by isolating the agent from patient data.
3. Context Management: Proposed and validated a sub-agent delegation strategy to mitigate "context rot," showing that offloading information retrieval to a subordinate agent significantly improves accuracy in long dialogues and reduces token consumption.
4. Knowledge Base Integration: Showed that access to external medical ontologies (UMLS) significantly improves accuracy for queries involving specialist terminology and brand-name-to-generic-name mapping.

Results

The system was evaluated on three datasets: the EpiCoho-M dataset (adapted from Ziletti and D'Ambrosi, 2025), a synthetic UK Medicine Brands dataset, and a Real World dataset derived from beta-testing user requests.

• Accuracy in Multi-Turn Dialogues: In chained (multi-turn) question scenarios, the single-agent configuration suffered significant accuracy degradation (F1-score dropped from 0.66 to 0.12 on the Real World dataset). The inclusion of the Retrieval Agent restored performance, achieving an F1-score of 0.73–0.75.
• Efficiency: The sub-agent approach reduced token consumption by approximately 93% (from ~855k to ~56k tokens) in chained scenarios.
• Knowledge Integration: On the UK Medicine Brands dataset, the configuration using the Retrieval Agent plus UMLS achieved an F1-score of 0.85, compared to 0.26 for the single-agent baseline without external knowledge.
• Operational Performance: The system successfully handled diverse workflows, including cohort listing, statistical summarization, and distribution analysis. Automated query generation took approximately 15 seconds for databases containing 16,452 concepts.

Significance and Claims

The paper claims that ClinQueryAgent serves as a viable exemplar for future AI-enabled text-to-query systems where data security is paramount. By shifting the verification burden from "trusting the LLM" to "verifying the logical implementation of the request" (e.g., "Diabetes II AND Semaglutide"), the system empowers clinicians and analysts to co-author queries without programming expertise. The authors posit that their multi-agent decomposition and logical query language are generic features that can be generalized to other healthcare contexts beyond the specific eHealthScope platform. The study concludes that such systems can effectively bridge the gap between complex clinical data and actionable population health insights while adhering to strict privacy constraints.