💻 computer science

World2Rules: A Neuro-Symbolic Framework for Learning World-Governing Safety Rules for Aviation

World2Rules is a neuro-symbolic framework that learns interpretable, verifiable safety rules from noisy multimodal aviation data by combining neural models for candidate generation with inductive logic programming and hierarchical reflective reasoning to ensure consistency, outperforming purely neural and baseline neuro-symbolic approaches in accuracy.

Original authors: Haichuan Wang, Jay Patrikar, Sebastian Scherer

Published 2026-04-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Haichuan Wang, Jay Patrikar, Sebastian Scherer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to fly a plane safely. You have two massive piles of information:

The "Normal Day" Pile: Thousands of videos and logs showing planes landing, taking off, and taxiing perfectly without any issues.
The "Oops" Pile: A small, messy stack of accident reports and near-miss stories where things went wrong.

The goal is to teach the robot the unwritten rules of the sky so it never crashes. This is exactly what the paper "World2Rules" is about.

Here is the story of how they did it, using some everyday analogies.

The Problem: The "Smart but Clumsy" Student

The researchers tried two main ways to teach the robot, and both had flaws:

The "Super-Reader" (Neural Models/LLMs): Imagine a student who has read every book in the library. They are great at spotting patterns in the "Oops" pile. If you show them a crash report, they can guess, "Oh, the plane was too close to the other one!"
- The Flaw: This student is a bit of a daydreamer. Sometimes they make things up (hallucinations), get confused by messy handwriting in the reports, or invent rules that sound smart but are actually dangerous. They can't prove why they are right, they just "feel" it.
The "Strict Lawyer" (Symbolic Logic/ILP): Imagine a lawyer who only accepts facts that are 100% proven and written in stone. They are perfect at checking if a rule is logically sound.
- The Flaw: This lawyer is terrible at reading messy handwriting. If you give them a blurry accident report with typos, they get stuck and refuse to learn anything because the data isn't "clean" enough.

The Solution: The "World2Rules" Team

The authors created a team-up between the Super-Reader and the Strict Lawyer. They call this a "Neuro-Symbolic" framework.

Think of it like a Detective Agency:

The Detective (The Neural Model): The Detective goes out into the messy world (the accident reports and video feeds). They gather clues, sketch out a theory of what happened, and propose a rule: "I think the rule is: If Plane A is landing and Plane B is on the runway, that's a crash!"
The Judge (The Logic Solver): The Judge takes the Detective's theory and puts it on trial. They check it against the "Normal Day" pile and the "Oops" pile.
- Does this rule break a safe flight? If yes, the Judge throws it out.
- Is this rule supported by enough evidence? If the Detective only guessed based on one weird accident, the Judge says, "Not enough proof."

The Secret Sauce: "Reflective Reasoning"

The magic of World2Rules is that they don't just do this once. They do it in four layers of checking, like a high-security airport screening process:

Level 1 (The ID Check): Before the Judge even looks at the theory, they check the Detective's notes. Is the handwriting legible? Did the Detective invent a plane that doesn't exist? If the notes are garbage, they are tossed immediately.
Level 2 (The Small Group Trial): They test the rule on just one accident and one normal day. Does the rule hold up for this specific pair? If the rule fails here, it's discarded.
Level 3 (The Jury Deliberation): This is the most important part. They take all the rules that passed the small trials and try to combine them. Imagine a jury trying to agree on a single verdict. If adding a new rule makes the whole group disagree (inconsistent), that rule is kicked out. They only keep the rules that work together perfectly for everyone.
Level 4 (The Final Polish): Finally, they look at the surviving rules. If a rule only applies to one very specific, weird accident (like "Planes crashing only on Tuesdays"), they delete it. They keep only the rules that happen often and matter a lot.

Why This Matters

In the real world, safety rules are messy. Accident reports are written by tired humans with typos. Normal flight data is boring and huge.

Without this system: A robot might learn a rule like "Planes crash when the sky is blue" because it saw one crash on a blue day. That's dangerous.
With World2Rules: The system filters out the noise. It learns rules like "Planes crash when two of them try to use the same runway at the same time."

The Results

When they tested this on real aviation data:

The "Super-Reader" alone got about 70% of the safety rules right.
The "Strict Lawyer" alone (without the Detective's help) got about 50% right because the data was too messy.
World2Rules (The Team) got 94% right!

The Takeaway

This paper shows that to build safe AI for critical jobs (like flying planes, driving cars, or running hospitals), you can't just rely on "gut feeling" AI. You need a system where the AI proposes ideas, but a logical "safety net" checks every single one of them against the rules of reality.

It's like having a brilliant but chaotic inventor working alongside a meticulous safety inspector. The inventor comes up with the ideas, and the inspector makes sure they actually work and won't blow up the factory. The result is a set of safety rules that are not only smart but also proven, clear, and trustworthy.

1. Problem Statement

The paper addresses the challenge of learning explicit, verifiable safety rules for real-world, safety-critical systems (specifically aviation) from multimodal data.

The Core Difficulty: Safety rules are often implicit in data, procedures, and regulations rather than explicitly defined. Learning them is complicated by:
- Data Asymmetry: Abundant "nominal" (safe) operational data vs. sparse, noisy "violation" (crash/incident) data.
- Neural Limitations: Pure neural models (LLMs/VLMs) extract patterns well but lack logical consistency, formal guarantees, and robustness against hallucinations or spurious correlations.
- Symbolic Limitations: Pure Inductive Logic Programming (ILP) is verifiable but brittle when applied directly to imperfect, noisy raw data extracted from unstructured sources.
Goal: To learn First-Order Logic (FOL) rules that define unsafe world configurations (e.g., potential collisions) by combining the pattern recognition of neural models with the rigorous verification of symbolic solvers.

2. Methodology: The World2Rules Framework

World2Rules is a neuro-symbolic framework that treats pretrained models as proposal mechanisms and ILP as a verification layer. It employs a hierarchical reflective reasoning process to filter noise and ensure consistency.

A. Data Extraction Pipeline

The system processes two distinct data sources using a four-step pipeline (Preprocessing, Extraction, Context Generation, Validation):

Violation Data (Positive Examples): Derived from unstructured aviation incident/crash reports (e.g., ASIAS system).
- Tool: Large Language Models (LLMs) extract typed entities (e.g., Aircraft, Runway) and relations.
Nominal Data (Negative Examples): Derived from routine airport surface operations (images + trajectory data).
- Tool: Vision-Language Models (VLMs) extract entities and relations directly from annotated images.
Output: Both streams generate Prolog-compatible inputs: Background Knowledge ( $B$ ), Positive Examples ( $E^+$ ), and Negative Examples ( $E^-$ ).

B. The Four-Level Feedback Mechanism

Instead of running ILP once on the entire dataset, World2Rules uses an iterative, solver-backed aggregation process to handle noise and inconsistency:

Level 1: Extraction Consistency: Validates that extracted Prolog bundles are syntactically and semantically valid before ILP learning begins. Invalid extractions are discarded or reprocessed.
Level 2: Subset-Level ILP Feedback:
- The data is paired into independent subsets (one violation report + one nominal observation).
- An ILP solver (Popper) runs on each subset independently.
- If a solver cannot find a hypothesis consistent with that specific subset (i.e., the data is fundamentally inconsistent), the subset is discarded.
Level 3: Iterative Global Aggregation:
- The system iteratively aggregates "reliable" subsets.
- It attempts to merge a new subset into the current aggregated hypothesis.
- Constraint: A subset is only accepted if the new aggregated hypothesis maintains 100% accuracy on the union of all accepted positive and negative examples.
- If a subset causes inconsistency, the system attempts to prune specific examples from that subset to restore consistency. If that fails, the subset is rejected.
- Robustness: To avoid order-dependency, the process includes a shuffle-based fallback mechanism if too many subsets are discarded.
Level 4: Rule-Level Support Pruning:
- After aggregation, rules are evaluated based on empirical support (how many positive examples they cover).
- Rules with low support (likely noise or overfitting) are pruned using a threshold ( $\tau = 0.20$ ).

C. Theoretical Guarantees

The paper provides proofs for Correctness and Safety:

Negative-Safety: The final hypothesis is guaranteed not to entail any negative example (no false positives regarding safety violations).
Training Correctness: Before pruning, the hypothesis covers all positive examples and rejects all negative examples.

3. Key Contributions

World2Rules Framework: A novel neuro-symbolic architecture that learns declarative safety rules from raw multimodal data (text reports + visual trajectories) without requiring clean, pre-labeled symbolic inputs.
Hierarchical Consistency Checking: A four-level feedback loop that integrates pretrained extractors with solver-backed verification, effectively filtering hallucinations and inconsistent evidence before rule induction.
Contrastive Learning Strategy: Explicitly leverages the complementary nature of nominal data (constraining permissible states) and violation data (delineating unsafe states) to learn robust rules.
Domain Application: Successfully instantiated for aviation surface operations, specifically runway incursion detection.

4. Experimental Results

The framework was evaluated on a held-out test set of 38 runway incursion scenarios (28 real-world, 10 canonical).

Performance Comparison:
- World2Rules: Achieved 94.0% F1 Score.
- LLM Only (GPT-4o): 70.4% F1 Score (23.6% lower).
- Naïve ILP (Single-pass): 50.8% F1 Score (43.2% lower).
Precision vs. Recall:
- All ILP variants achieved near-perfect Precision (avoiding false alarms), demonstrating the solver's ability to prevent spurious rules.
- The primary gain for World2Rules was in Recall, driven by the iterative aggregation mechanism which successfully captured diverse incursion patterns that single-pass methods missed.
Data Scaling: Performance improved monotonically as the number of violation reports increased (from 10 to 300 reports), with F1 rising from 52.5% to 94.0%, proving the system scales effectively with more diverse training data.
Qualitative Output: The system learned compact, interpretable rules (e.g., "Collision occurs if Aircraft A is landing on Runway X while Aircraft B is holding on Runway X") that align with known aviation safety categories.

5. Significance and Conclusion

Safety-Critical Reasoning: The framework bridges the gap between the flexibility of deep learning and the rigor required for safety certification. It produces rules that are not only predictive but formally verifiable and human-interpretable.
Robustness to Noise: By treating neural extractions as proposals subject to strict logical verification, the system mitigates the risks of hallucination and data imbalance common in safety-critical domains.
Generalizability: While tested on aviation, the approach is domain-agnostic. It can be applied to any safety-critical system governed by structured constraints, provided a predicate vocabulary and multimodal data sources are available.

In summary, World2Rules demonstrates that combining neural perception with iterative, solver-backed symbolic induction yields safety rules that are significantly more accurate, robust, and reliable than purely neural or naive neuro-symbolic approaches.