CIViC MCP: Integrating Large Language Models with the Clinical Interpretations of Variants in Cancer

This paper introduces the CIViC MCP server, a tool that integrates large language models with the CIViC knowledgebase to enable users to query cancer variant data and receive rapid, natural language summaries of expertly curated clinical interpretations.

The Gist

Imagine you are a doctor trying to treat a patient with a rare form of cancer. You know the patient has a specific genetic mutation (a "typo" in their DNA), but you need to know: Does this typo make the cancer grow faster? Is there a specific drug that will kill it? What do the latest studies say?

The problem is that the answers are buried in thousands of scientific papers, databases, and complex charts. It's like trying to find a specific needle in a haystack, but the haystack is made of books, and the needles are made of tiny, shifting data points.

This is where the CIViC MCP project comes in. Here is a simple breakdown of what the paper is about, using some everyday analogies.

1. The Problem: The "Library of Babel"

Think of CIViC (Clinical Interpretations of Variants in Cancer) as the world's most organized, expert-curated library for cancer genetics. It contains thousands of "Evidence Items"—little cards written by scientists that say things like, "This specific gene mutation responds well to Drug X."

However, this library is huge and complex. If you ask a standard AI (a Large Language Model or LLM) a question like, "Does mutation Y work with Drug Z?", the AI has two bad options:

• The "Guessing Game": The AI tries to remember what it learned during its training. But since medical data changes fast, the AI might be outdated or just make up a fact (a "hallucination") because it's trying to be helpful.
• The "Web Surfer": The AI tries to browse the internet like a human, clicking links and reading pages. This is slow, clumsy, and often leads to dead ends or the wrong pages.

2. The Solution: The "Super-Concierge" (The MCP Server)

The authors built a new tool called the CIViC MCP Server. Think of this as a Super-Concierge who lives inside the library and speaks the AI's language perfectly.

Instead of letting the AI wander around the library or guess from memory, the AI now has a direct phone line to this Concierge.

• How it works: When you ask the AI a question, the AI doesn't try to write a complex search query itself (which it often gets wrong). Instead, it asks the Concierge: "Hey, I need to know about Mutation Y and Drug Z."
• The Magic: The Concierge knows exactly where the cards are. It pulls the exact data, checks the spelling (because scientists sometimes use different names for the same thing), and hands the AI a neat, organized summary with the original source citations.

3. Why "Predefined Tools" Matter

The paper mentions a clever trick they used. They realized that if they let the AI write its own search queries, it would often get the grammar wrong and fail.

So, they created pre-made "buttons" (tools) for the AI to press.

• Analogy: Imagine a restaurant menu. Instead of letting the customer write their own recipe from scratch (which might result in a disaster), the chef gives them a menu with specific, tested dishes. The customer just points to "The Burger" or "The Salad."
• In this case, the "buttons" are tools to look up Evidence (the raw study data) or Assertions (the expert summary). This ensures the AI never asks for something impossible and always gets the right data.

4. The Results: Speed and Accuracy

The team tested this new system against the old ways (AI guessing or AI browsing the web).

• Accuracy: The AI with the "Concierge" (MCP) got the answer right 95% of the time. The AI without it only got it right 30% of the time.
• Speed: The "Web Surfer" AI took about 7 minutes to find an answer because it was clicking through pages. The "Concierge" AI took about 43 seconds.
• Trust: Every answer the AI gives now comes with a direct link to the original scientific card, so a doctor can verify it instantly.

The Bottom Line

This paper describes a bridge between human-curated medical knowledge and AI intelligence.

Before, AI was like a student who had read a textbook five years ago and was trying to guess the answers to today's test. Now, with the CIViC MCP, the AI is like a student who has a direct line to the professor's office, can ask specific questions, and gets the exact, up-to-date answer with the source right there.

This means doctors and researchers can ask complex questions in plain English and get reliable, citation-backed answers in seconds, helping them make better decisions for cancer patients faster.

Technical Summary

1. Problem Statement

Precision oncology relies on the accurate interpretation of molecular variants to guide clinical decision-making. The Clinical Interpretations of Variants in Cancer (CIViC) knowledgebase is a critical, community-driven resource that curates evidence linking variants to clinical significance (e.g., diagnostic, predictive, prognostic). However, accessing this structured data presents challenges for Large Language Models (LLMs):

• Knowledge Gaps: LLMs trained on static datasets cannot access rapidly updated, specialized resources like CIViC, leading to outdated or hallucinated information.
• Retrieval Limitations: Standard web-based retrieval (search engines) is often imprecise for highly structured databases.
• Agent Inefficiency: "Agentic" approaches where LLMs simulate human browsing of web interfaces are slow, error-prone, and lack the fidelity of direct API access.
• Schema Complexity: Even with API access, LLMs struggle to correctly construct complex queries (e.g., GraphQL) without predefined tools, often misinterpreting domain-specific concepts like "Molecular Profiles" or "Assertions."

2. Methodology

The authors developed the CIViC Model Context Protocol (MCP) server, a standardized interface that bridges LLMs and the CIViC GraphQL API.

• Architecture & Tools:

  • The server is hosted on Cloudflare and exposes two primary tools to the LLM: get_variant_evidence (for individual Evidence Items) and get_variant_assertion (for higher-level summaries).
  • Predefined Queries: Instead of allowing the LLM to write raw GraphQL queries (which led to errors in early testing), the MCP server uses predefined, fixed-schema queries. The LLM only needs to supply parameters (Molecular Profile, Cancer Type, Therapy), ensuring query reliability.
  • Input Normalization: To handle spelling variations and synonyms, the server normalizes user inputs to CIViC's "preferred labels" using precomputed alias lists. It utilizes the VICC Gene Normalization Service for genes, CIVIc-curated lists for variants, the Disease Ontology for cancer types, and the NCI Thesaurus for therapies. Matching is performed using the Dice-Sørensen coefficient on character bigrams for speed and efficiency.
  • Data Enrichment: Responses include structured records (Evidence Type, Significance, Direction, PubMed IDs, URLs) and field definitions to aid the LLM in accurate summarization.

• Evaluation Benchmark:

  • Task: A controlled classification task where the LLM determines if a specific clinical significance is supported, contradicted, or unsupported by evidence for a given triplet (Molecular Profile, Cancer Type, Therapy).
  • Dataset: 100 randomly selected CIViC triplets.
  • Comparators:
    1. GPT-5 (Baseline): Zero-shot prompting without external tools.
    2. GPT-5 + Agent Mode: LLM simulates browsing the CIViC website.
    3. GPT-5 + MCP: LLM uses the CIViC MCP server tools.

3. Key Contributions

• CIViC MCP Server: The first implementation of an MCP server for a major oncology knowledgebase, enabling seamless, structured API access for LLMs.
• Hybrid Retrieval Strategy: A novel approach combining LLM natural language understanding with predefined, validated API queries to overcome the limitations of both pure LLM generation and unstructured web browsing.
• Robust Normalization Pipeline: A system for mapping diverse user inputs to canonical database identifiers using external ontologies and efficient string matching.
• Open Source Implementation: The server, Python scripts, and a public Chatbot interface are made available to the community to facilitate reproducible research and clinical decision support.

4. Results

The benchmarking results demonstrated that the MCP approach significantly outperformed other methods in both accuracy and efficiency:

• Accuracy:
  • GPT-5 + MCP: Achieved 0.95 overall accuracy and a 0.98 weighted F1 score.
  • GPT-5 + Agent Mode: Achieved 0.83 accuracy and 0.91 F1 (a marked improvement over the baseline but inferior to MCP).
  • GPT-5 (Baseline): Achieved only 0.30 accuracy and 0.46 F1, highlighting the inability of raw LLMs to access specialized, up-to-date data.
• Efficiency (Latency):
  • GPT-5 + MCP: ~43.1 seconds (95% CI: 39.7–47.0).
  • GPT-5 (Baseline): ~42.9 seconds.
  • GPT-5 + Agent Mode: ~425.0 seconds (95% CI: 378.6–476.4).
  • Conclusion: The MCP approach provides the accuracy of structured retrieval with the speed of direct generation, while Agent Mode is roughly 10 times slower.

5. Significance

• Clinical Reliability: By grounding LLM responses in real-time, curated data from CIViC, the system mitigates hallucination risks and ensures that clinical summaries are citation-rich and verifiable.
• Scalability: The MCP framework offers a blueprint for integrating other biomedical databases (e.g., ClinVar, OncoKB, DGIdb) into a unified conversational interface, moving beyond simple search to complex, multi-step data synthesis.
• Future Directions: The authors propose extending this work to include constrained query-composition methods, allowing LLMs to safely construct custom queries for unanticipated user questions while maintaining the reliability of the structured backend.

In summary, the CIViC MCP server represents a significant advancement in Agentic AI for biomedicine, proving that standardized, structured tool integration is superior to both raw LLM generation and web-browsing agents for accessing complex, high-stakes clinical knowledge.