Technical Summary: A Framework for Dynamic Knowledge Graph Ontology Incremental Alignment based on Large Language Model Agents

Problem Statement

Dynamic knowledge graphs (KGs) used in domains such as risk management, compliance monitoring, and industrial safety evolve continuously through the arrival of new entities, revised rules, and streaming events. However, existing ontology alignment methods face a persistent trade-off between semantic precision and update latency. Conventional static alignment methods require re-running the entire alignment pipeline after every update, which is computationally costly and unsuitable for streaming environments. Traditional incremental approaches reduce recomputation but often fail to interpret rich textual descriptions or semantic context changes, focusing primarily on structural updates. Furthermore, while Large Language Models (LLMs) have improved semantic matching, most current LLM-based methods are designed for static or batch alignment rather than persistent, continuous graph evolution. This gap necessitates a solution that can maintain semantic consistency in continuously evolving graphs without sacrificing efficiency or accuracy.

Methodology: DOKA-Agent

The paper proposes DOKA-Agent, a framework for dynamic ontology incremental alignment that utilizes a Large Language Model (LLM) agent operating within a perceive-plan-act workflow. The framework is designed to capture graph changes, enrich semantic context, and produce validated alignment decisions in real-time.

Core Architecture

The framework consists of three primary modules:

1. Dynamic Perception Module:

   • Input: A stream of raw database events ($E_{raw}$) containing operation metadata, timestamps, and changed graph objects.
   • Function: Translates low-level operations into structured semantic events (e.g., entity creation, relation modification). It serializes these into task descriptions ($T$) containing entity IDs, types, and structural details.
   • Quality Control: Measures event coverage and task completeness. If a task lacks sufficient attributes or neighborhood context, the module invokes a Graph Query Tool to supplement missing information before forwarding the task to the agent.

2. Agent Decision-Making Core:

   • Planning Phase: The LLM decomposes the incoming task into executable subtasks ($P$), adapting the tool invocation order based on event type and semantic complexity. It queries the graph context ($C$) to retrieve descriptive text, neighboring entities, and relations.
   • Action Phase:
     • Encoding: The LLM generates a unified semantic description ($d'$) and embedding ($v$) from the enriched context.
     • Hybrid Retrieval: Candidates are retrieved from a vector index using a dense-sparse strategy.
     • Hybrid Similarity Scoring: For each candidate, the system computes a weighted similarity score combining:
       • Semantic Similarity: Based on LLM-generated embeddings.
       • Structural Similarity: Calculated via the intersection over union of relation sets ($|R \cap sR,i| / |R \cup sR,i|$).
       • Lexical Similarity: Estimated using normalized edit distance.
     • Relation Decision: The LLM evaluates the candidates and their similarity scores to predict the relation type (e.g., equivalent, broader, narrower) and a confidence score ($\theta$).
   • Commitment: The system selects the highest-confidence decision. Alignments are committed to the graph only if the confidence exceeds a threshold ($\theta_{thresh} = 0.7$); low-confidence matches are rejected.

3. Tools and Knowledge Base:

   • The framework relies on a Graph Query Tool for neighborhood context extraction and a Vector Search Tool for efficient similarity search over an incremental vector index.
   • A dual-layer knowledge base (Graph DB and IndexDB) supports semantic retrieval and persistent graph updates.
   • Optimization: The hybrid similarity weights are optimized using a total loss function combining cross-entropy and L2 regularization to balance semantic, structural, and lexical contributions.

Key Contributions

The paper outlines three specific contributions:

1. DOKA-Agent Framework: A dynamic ontology alignment framework specifically tailored for evolving knowledge graphs, moving beyond static snapshots.
2. Incremental Perceive-Plan-Act Workflow: A novel workflow integrating context completion, hybrid retrieval (dense-sparse), and relation-aware decision-making, allowing the agent to adapt to event types and semantic complexity.
3. Empirical Validation: Comprehensive validation on dynamic knowledge graph benchmarks demonstrating improvements in both alignment quality and operational efficiency compared to existing static and dynamic baselines.

Experimental Results

The framework was evaluated on DBP15K dynamic partitions (ZH-EN, JA-EN, FR-EN), simulating graph evolution across three time steps ($t_0, t_1, t_2$).

• Static Alignment Performance: DOKA-Agent achieved the best overall performance across all subsets, outperforming strong baselines like DINGAL-B, HGCN-JE, and LLM-Static. For example, on DBP15K-FR-EN, it achieved a Hit@1 of 93.5% and F1 of 96.2%, surpassing the next best method (LLM-Static) by 0.9% in Hit@1.
• Dynamic Alignment Performance: In incremental settings ($t_1, t_2$), DOKA-Agent consistently outperformed dynamic baselines (DINGAL-O, DINGAL-U) and direct LLM approaches (LLM-Direct). The results suggest that explicitly capturing change events and re-estimating relations with updated context is more effective than purely structural updates or direct matching.
• Operational Efficiency: DOKA-Agent demonstrated significant efficiency gains. On the DBP15K-ZH-EN dataset at $t_1$, it achieved an average processing time of 0.9 seconds per event and a total alignment time of 3.8 seconds, representing a 96.3% efficiency improvement over the static baseline DINGAL-B (102.5 seconds).
• Ablation Study: Removing core components (Planning Module, Semantic Similarity, or Information Completion) resulted in measurable performance drops, confirming that the integration of planning, semantic encoding, and context enrichment is essential for the framework's success.

Significance and Claims

The paper claims that agentic orchestration provides a practical mechanism for maintaining semantic consistency in continuously evolving knowledge graphs. By framing ontology alignment not as a monolithic prediction step but as a dynamic workflow involving perception, planning, and tool use, DOKA-Agent addresses the limitations of both traditional incremental methods (which lack semantic depth) and static LLM methods (which lack responsiveness to streaming changes).

The authors emphasize that this approach offers a flexible interface for future extensions, such as stronger constraint checking and domain-specific adaptation for sectors like industrial safety and education. However, the paper maintains a modest stance regarding limitations, acknowledging that the current implementation depends on the quality of graph-side event capture and the stability of LLM outputs. It notes that LLM-based matching remains vulnerable to hallucinations and that the evaluation on DBP15K does not fully cover domain-specific constraints or real-world industrial update noise. Future work is suggested to explore multi-agent collaboration and larger streaming benchmarks.