Building Trust in the Skies: A Knowledge-Grounded LLM-based Framework for Aviation Safety

1. Problem Statement

The integration of Large Language Models (LLMs) into aviation safety decision-making faces critical barriers due to the industry's "safety-critical" nature. While LLMs offer potential for automating the analysis of incident reports and regulatory documents, they suffer from inherent limitations that make standalone deployment dangerous:

• Hallucinations and Factual Inaccuracies: LLMs often generate plausible but incorrect information, such as citing non-existent regulations or misinterpreting aircraft system limitations.
• Lack of Verifiability: LLMs operate as "black boxes" without explicit linkage to authoritative sources, violating aviation standards (e.g., DO-178C, ISO/IEC/IEEE 8800) that require traceability, auditability, and provenance.
• Static Knowledge: Standard LLMs have temporal knowledge cutoffs and cannot dynamically incorporate real-time data like new Airworthiness Directives (ADs) or recent incident bulletins.
• Limitations of Existing Solutions: Traditional Knowledge Graphs (KGs) offer structure and verifiability but are expensive to build manually and lack adaptability. Conversely, standard Retrieval-Augmented Generation (RAG) retrieves unstructured text chunks rather than structured facts, potentially propagating errors from source documents.

2. Methodology: The Integrated Pipeline

The authors propose a novel, end-to-end dual-phase framework that synergistically combines LLMs and Knowledge Graphs (KGs) to create a closed-loop, self-reinforcing system. The architecture is built on a GraphRAG approach, specifically tailored for aviation safety.

Phase 1: Automated Knowledge Graph Construction (ASKG)

• Data Ingestion: The system processes heterogeneous multimodal sources, including NTSB accident reports (1990–2025), FAA registration databases, and airline operational records.
• Entity Extraction & Resolution:
  • Utilizes LangChain for orchestration, spaCy for domain-adapted Named Entity Recognition (NER), and SentenceTransformers with FAISS for entity resolution.
  • Implements a three-tier resolution strategy to handle terminology ambiguity (e.g., distinguishing "B737-800" from "Boeing 737-800") via lexical normalization, embedding similarity, and rule-based processing.
• Ontology & Schema:
  • Built upon a foundational Aviation Safety Ontology defining 7 entity types (Agents, Conditions, Facilities, Locations, Operations, Organizations, Vehicles) and 5 relationship types.
  • Data is stored in Neo4j, enforcing strict schema constraints (unique constraints on registration numbers/ICAO codes) and using composite indexes for performance.
• Human-in-the-Loop: Initial training data was manually annotated (Cessna/NTSB summaries) to train a machine learning model (TF-IDF + Logistic Regression) for scaling annotation across 10,000+ records.

Phase 2: Knowledge-Grounded Reasoning (GraphRAG)

• Natural Language to Cypher (Text2Cypher):
  • Instead of standard vector search, the system uses an LLM (GPT-3.5 or local Llama-3) to translate natural language queries into optimized Cypher graph queries.
  • Few-shot learning with aviation-specific prompt templates ensures schema compliance.
  • A Context Manager maintains conversation history to resolve multi-turn query ambiguities.
• Execution & Optimization:
  • Queries are executed against the Neo4j graph.
  • Semantic Caching (Redis) stores time-bounded query results to reduce latency for repeated queries.
  • Query Plan Analysis detects missing indexes, and automatic pagination prevents resource exhaustion.
• Response Generation: The retrieved sub-graph clusters (structured facts) are fed back to the LLM to generate a final natural language response. This ensures the output is strictly grounded in the retrieved evidence, mitigating hallucinations.

3. Key Contributions

• Novel Hybrid Architecture: The paper introduces a tightly coupled, bidirectional pipeline where LLMs dynamically construct/maintain the KG, and the KG rigorously validates LLM outputs. This bridges the gap between the flexibility of LLMs and the determinism of KGs.
• Aviation Safety Ontology (ASKG): The creation of a specialized, structured ontology for aviation safety that encodes causal relationships between hazards, failures, regulations, and outcomes.
• Text2Cypher for Safety: A specialized translation layer that converts natural language safety inquiries into precise graph queries, outperforming traditional keyword-based or pure vector-search RAG approaches in handling complex, multi-hop reasoning.
• Traceability and Explainability: The system provides a transparent link between user queries and the underlying structured evidence (graph nodes), satisfying regulatory requirements for auditability.

4. Results and Implementation

• Dataset: The system was implemented using 68,681 preprocessed NTSB accident records (1990–2025) with 38 features.
• Scale: The resulting Knowledge Graph contains 205,922 entities and 137,241 relationships.
• Performance:
  • The GraphRAG approach demonstrated superior accuracy in Schema Accuracy (mapping to NTSB hierarchy), Query Precision (generating valid Cypher), and Contextual Grounding (verifying responses against retrieved nodes).
  • It successfully handled complex multi-hop queries (e.g., correlating environmental conditions with incident types across airline fleets) with performance largely independent of traversal depth.
• System Demo: A Flask-based web interface (Fig. 4 & 5) allows non-technical users to perform complex relational searches. The system visualizes the translation of natural language to Cypher and displays synthesized insights alongside system metrics.
• Open Source: The complete pipeline code is publicly available on GitHub.

5. Significance and Future Directions

• Significance: This work addresses the "trust gap" in AI for aviation. By grounding LLMs in a verifiable, structured Knowledge Graph, the framework enables the safe adoption of AI for regulatory compliance, accident investigation, and predictive risk assessment. It moves beyond "black box" recommendations to "explainable" insights.
• Limitations: Current limitations include reliance on LLMs for Text2Cypher (which can produce semantically incomplete queries in complex scenarios) and a lack of multimodal data integration (e.g., cockpit audio, telemetry).
• Future Work:
  • Transitioning from traditional ML classifiers to LLM-based NER for finer relationship extraction.
  • Implementing Vector-Graph Hybrid Retrieval to search both structured graphs and raw text simultaneously.
  • Developing a collaborative feedback loop where safety experts can manually correct or "upvote" generated queries to fine-tune the model.

In conclusion, the paper presents a robust, scalable framework that transforms aviation safety analytics from unstructured text processing into a verifiable, graph-grounded decision support system, setting a new standard for trustworthy AI in safety-critical domains.