Large Language Model Integration for Knowledge Retrieval and Interaction for the DUNE Experiment

This paper introduces DUNE-GPT, a prototype framework that utilizes large language models and retrieval-augmented generation to enable secure, natural-language querying of the Deep Underground Neutrino Experiment's extensive documentation and technical resources, thereby enhancing collaboration productivity and onboarding.

The Gist

Imagine the DUNE experiment as a massive, bustling library dedicated to understanding the universe's most elusive particles (neutrinos). This library is so huge that it contains millions of books, notes, blueprints, and meeting minutes scattered across different shelves, some in digital formats and some in physical binders. For a new librarian (or a scientist) trying to find a specific detail about how a detector works, searching through this maze can take hours or even days.

The paper introduces DUNE-GPT, a new "super-librarian" designed to solve this problem. Here is how it works, broken down into simple concepts:

1. The Problem: A Library Too Big to Search

The DUNE collaboration has so much information stored in different places (like DocDB and Indico) that finding specific technical answers is difficult. It's like trying to find a single sentence in a library where the books are unorganized and the catalog is broken.

2. The Solution: A Smart, Internal Assistant

The team built DUNE-GPT, a prototype tool that acts like a knowledgeable guide. Instead of making you search through folders, you can just ask it a question in plain English, like, "How does the reconstruction algorithm handle noise?" or "Where are the meeting notes from last Tuesday?"

3. How It Works: The "Three-Step Detective"

The system doesn't just guess; it follows a strict three-step process to ensure accuracy:

• Step 1: Reading and Indexing (The Librarian's Brain):
  First, the system reads through all the allowed documents (technical reports, meeting notes, etc.). It breaks them down into tiny pieces and creates a "mental map" (called an embedding) of what each piece is about. Think of this as the librarian reading every book and writing a summary card for every page.

  • Note: They are very careful to only read documents that are safe for everyone to see, leaving out any secret or restricted files.

• Step 2: The Quick Search (The Vector Database):
  When you ask a question, the system doesn't read the whole library again. Instead, it uses a high-speed search engine (called FAISS) to instantly find the specific pages in its "mental map" that match your question. It's like the librarian instantly pulling the three most relevant books off the shelf based on your query.

• Step 3: The Answer (The AI Writer):
  The system takes those specific pages it found and hands them to a Large Language Model (the "AI writer"). The AI reads only those pages and writes an answer for you.

  • Crucial Safety Feature: The AI is told, "You must answer based only on these pages." This prevents the AI from making things up (a problem called "hallucination") and ensures the answer is grounded in real DUNE facts.

4. Safety and Privacy: The "Walled Garden"

One of the biggest worries with AI is leaking private data to the public internet. To fix this, the DUNE team built this system entirely inside their own secure computer network (at Fermilab and Argonne).

• It's like building the library inside a secure fortress.
• Only people with a key (authenticated DUNE collaborators) can enter.
• No data leaves the fortress to go to public AI companies.

5. What They Found So Far

The team tested this prototype and found that it is quite good at its job.

• Accuracy: In early tests, it successfully found the right documents about 70% of the time, even for tricky questions about detector details or physics workflows.
• Interface: They built a simple website where scientists can type questions and get answers that include links back to the original documents, so you can verify the source.

6. What's Next?

The tool is still a prototype (a "beta" version). The team plans to:

• Teach it to read more types of files, like code and detector logs.
• Make it understand complex charts and graphs (images).
• Roll it out to the entire collaboration for everyone to use.

In summary: DUNE-GPT is a secure, internal search engine that uses AI to help scientists find answers in their massive library of documents quickly and accurately, without ever leaving their secure network or risking data privacy.

Technical Summary

Technical Summary: DUNE-GPT – Large Language Model Integration for Knowledge Retrieval and Interaction

Problem Statement
The Deep Underground Neutrino Experiment (DUNE) faces significant challenges in knowledge management due to the unprecedented volume and heterogeneity of its data. Information is distributed across multiple platforms, including DocDB, Indico, and internal wikis, encompassing documentation, technical notes, experimental data, and reconstruction pipelines. As the collaboration grows, navigating this distributed landscape to access specific technical details regarding reconstruction, simulation, and detector operations becomes increasingly time-consuming, particularly for new collaborators. Furthermore, the direct deployment of commercial or open Large Language Models (LLMs) in High-Energy Physics (HEP) collaborations raises critical concerns regarding data privacy, reproducibility, and network accessibility, necessitating a secure, internal solution.

Methodology
To address these challenges, the authors developed DUNE-GPT, a prototype framework operating within Fermilab's computing environment. The system utilizes a Retrieval-Augmented Generation (RAG) architecture, consisting of three primary components:

1. Document Processing and Embedding Generation: The system ingests publicly accessible and internal DUNE documentation (e.g., technical design reports, meeting notes, working group materials) from DocDB and Indico. Text preprocessing involves parsing various formats (PDF, DOCX, TXT, PNG), extracting metadata, and performing token-level segmentation. Sensitive or restricted content is excluded to comply with collaboration policies. Processed documents are embedded using transformer-based encoders (specifically multi-qa-mpnet-base-dot-v1), optimized for scientific text.
2. Vector Database and Retrieval Engine: The generated embeddings are stored in a vector database using Facebook AI Similarity Search (FAISS). Upon receiving a user query, the system performs a similarity search using cosine similarity to retrieve the most relevant text segments.
3. LLM Interface for Response Generation: The retrieved context snippets are forwarded to pre-existing LLMs hosted on Argonne (Argo) and Fermilab (Ollama) machines. The LLM is conditioned on these verified source snippets to generate responses. This approach ensures that outputs remain grounded in DUNE-specific sources, minimizing the risk of "hallucination" while maintaining data privacy by keeping all operations within the internal computing environment.

Key Contributions and Implementation

• Secure Internal Architecture: Unlike external AI tools, DUNE-GPT is designed to operate entirely within Fermilab's and Argonne's computing resources, ensuring that only authenticated DUNE collaborators can access the tool and that no sensitive data leaves the internal network.
• Citation-Enabled Interface: The system provides a lightweight web interface where users can submit natural-language queries. The generated responses are formatted to include citations referencing the specific documents retrieved from DUNE internal databases, enhancing traceability.
• Prototype Deployment: The prototype was initially developed using the Aurora supercomputer at the Argonne Leadership Computing Facility (ALCF) with a Python backend integrating open-source libraries. The long-term plan involves migrating the infrastructure to Fermilab for collaboration-wide hosting.

Preliminary Results
Initial benchmarks indicate that the RAG-based system achieves an accuracy of approximately 70% in retrieving relevant documentation across diverse query types, including detector specifics, reconstruction algorithms, and physics analysis workflows. The current frontend demonstrates the ability to display a sample question, the generated response, and the top three retrieved references used in the generation process.

Significance and Future Outlook
The paper positions DUNE-GPT as a critical step toward improving collaboration-wide productivity and onboarding efficiency by enabling natural-language querying of complex technical resources. The authors maintain a modest scope, noting the tool is currently under active development. Future work focuses on:

• Expanding the embedding corpus to include detector operation logs and code documentation.
• Developing a domain-adapted LLM specifically for HEP language understanding.
• Integrating multi-modal content (e.g., plots and figures).
• Establishing web hosting at Fermilab to facilitate secure, collaboration-wide access.
• Incorporating user feedback to refine retrieval quality and interface usability.

The work represents a controlled, explainable approach to integrating AI into HP workflows, prioritizing data security and source verification over the deployment of unverified external models.