🤖 machine learning

Logic of Hypotheses: from Zero to Full Knowledge in Neurosymbolic Integration

This paper introduces Logic of Hypotheses (LoH), a novel neurosymbolic framework that unifies hand-crafted rules and data-driven learning by extending propositional logic with a learnable choice operator compiled into differentiable fuzzy logic, enabling flexible knowledge integration and discrete Boolean reasoning without performance loss.

Original authors: Davide Bizzaro, Alessandro Daniele

Published 2026-05-18

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Davide Bizzaro, Alessandro Daniele

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Bridging Two Worlds

Imagine you are trying to teach a computer to make smart decisions. You have two main tools:

The Neural Network (The "Intuitive Learner"): This is like a child who learns by looking at thousands of pictures. It's great at spotting patterns (like recognizing a cat in a photo) but terrible at explaining why it thinks something is a cat. It's a "black box."
Symbolic Logic (The "Rule-Follower"): This is like a strict lawyer or a chess grandmaster. It follows clear, written rules (e.g., "If it rains, take an umbrella"). It is perfect at explaining its reasoning but terrible at learning new things from raw data without being told exactly what to do.

Neurosymbolic Integration (NeSy) is the attempt to marry these two. The goal is to build a system that can learn from data and follow logical rules. However, most current methods are stuck at the extremes: either you feed the computer perfect rules (and it just learns to follow them), or you let it learn rules from scratch (and it often makes up nonsense).

The Problem: What if you have some rules, but they are incomplete? Or what if you have a list of possible rules but aren't sure which ones are correct? Current tools struggle to handle this "middle ground."

The Solution: Logic of Hypotheses (LoH)

The authors introduce a new language called Logic of Hypotheses (LoH). Think of LoH as a "Choose Your Own Adventure" book for computer logic.

The "Choice Operator"

In standard logic, a rule is fixed. In LoH, you can insert a special "choice box" into a rule.

Standard Logic: "If it rains, take an umbrella." (Fixed)
LoH: "If it rains, take [an umbrella OR a raincoat OR a hood]."

The computer doesn't know which one to pick yet. It has to look at the data and decide: "Based on the pictures I've seen, 'raincoat' seems to work best."

This allows the system to handle three different scenarios seamlessly:

Full Knowledge: You give the computer the exact rule. The "choice box" is empty because there's only one option. The computer just learns to recognize the rain.
Zero Knowledge: You give the computer a giant list of every possible rule it could learn. It has to figure out the whole structure from scratch.
Partial Knowledge (The Sweet Spot): You give the computer a skeleton of the rule and a few options for the missing parts. It fills in the blanks based on the data.

How It Works: The "Gödel Trick"

Turning these "choice boxes" into a computer program that can learn is tricky. Usually, computers need smooth, sliding scales (like a dimmer switch) to learn. But logic is binary (on/off, yes/no).

The authors use a clever mathematical trick involving Gödel fuzzy logic.

The Analogy: Imagine a dimmer switch that controls a light. Usually, you can set it to 30% or 70%. But for logic, you need the light to be either fully ON or fully OFF.
The Trick: The authors designed the system so that even though the computer is learning with smooth, sliding switches (using gradients), it is mathematically guaranteed that when you snap the switch to "ON" or "OFF" at the end, the answer is exactly the same as if it had been learning with hard switches the whole time.
Why this matters: Many other systems lose accuracy when they try to turn their smooth learning into hard, readable rules. This system does it without losing any performance. It's like a chef who can cook a perfect meal using a precise scale, but can also serve it with a rough estimate of the ingredients, and the taste is identical.

The Experiments: Does It Work?

The authors tested this "Choose Your Own Adventure" logic on two types of tasks:

1. Tabular Data (The "Spreadsheet" Test)
They used standard datasets (like predicting if a bank loan should be approved).

Result: Their model performed just as well as the best "black box" neural networks but produced clear, readable rules that humans could understand. It beat other rule-learning systems that often get confused when trying to be both flexible and accurate.

2. Visual Tic-Tac-Toe (The "Perception" Test)
This was a harder test. Instead of giving the computer a board with "X" and "O" written on it, they gave it images of the board (using handwritten digits to represent X, O, and empty spaces).

The Challenge: The computer had to first learn to recognize the digits (perception) and then learn the rules of Tic-Tac-Toe (logic) all at once.
Result: The LoH model successfully learned the rules of the game directly from the images. It figured out that "three X's in a row means a win" and could explain this rule in plain English. Other models either failed to learn the rules or learned rules that didn't make sense when you tried to read them.

The Wildfire Example

To show how flexible this is, they simulated a wildfire risk assessment.

Scenario A (Full Knowledge): They told the computer the exact physics of fire (Fuel + Dryness + Trigger). The computer just learned to recognize "forest" and "dry vegetation" from satellite images.
Scenario B (Partial Knowledge): They told the computer, "We know how Fuel works, but we aren't sure about the Trigger." The computer was given a list of 5 possible "Trigger" rules. It looked at the data and picked the one that actually predicted fires best.
Scenario C (Zero Knowledge): They gave the computer a list of 20 possible rules for everything and let it figure out the whole system from scratch.

In every scenario, the system adapted perfectly, proving it can handle anything from "I know everything" to "I know nothing."

Summary

The paper presents Logic of Hypotheses (LoH), a new way to build AI that sits comfortably between "rigid rules" and "blind learning."

It uses choice boxes to let the AI decide which rules to use.
It uses a mathematical trick to ensure that the rules it learns are perfectly accurate, even after being simplified for humans to read.
It works on both spreadsheets and images, proving it can handle complex real-world problems where we might only have partial knowledge.

Essentially, LoH gives AI the ability to say, "I know the general shape of the answer, but I need to look at the data to fill in the specific details," and then do so with perfect accuracy.

Technical Summary: Logic of Hypotheses (LoH)

Problem Statement

Neurosymbolic integration (NeSy) aims to combine the deductive power of symbolic logic with the learning capabilities of neural networks. Existing approaches typically occupy opposite ends of a spectrum:

Knowledge Injection: Methods like DeepProbLog and LTN rely on domain experts providing high-quality logic formulas as priors or constraints. This assumes such rules are readily available and does not handle scenarios where rules must be learned or partially revised.
Rule Induction: Methods like SATNet, DSL, and Inductive Logic Programming (ILP) learn symbolic rules directly from data. While flexible, they often struggle to incorporate partial prior knowledge or enforce specific syntactic templates (e.g., fixed-length clauses) without ad-hoc machinery.

A significant gap remains in handling intermediate scenarios where partial logical structure is supplied, but missing components must be learned, or where learned rules must adhere to specific syntactic constraints. Current differentiable NeSy models often lack the flexibility to span this "zero to full knowledge" spectrum within a single, principled framework. Furthermore, many existing models using fuzzy logic (e.g., Product or Łukasiewicz t-norms) suffer from a discrepancy between their continuous training performance and their discrete, interpretable rule extraction, leading to accuracy loss upon binarization.

Methodology: The Logic of Hypotheses (LoH)

The authors propose Logic of Hypotheses (LoH), a novel logical formalism that unifies knowledge injection and rule induction.

1. The Choice Operator

LoH extends standard propositional logic syntax with a choice operator $[\Phi_1, \dots, \Phi_n]$ .

Syntax: A formula $\Phi$ can contain choice operators that select one subformula from a finite pool of candidates.
Semantics: A single LoH formula represents a hypothesis space $\mathcal{H}(\Phi)$ , which is the set of all classical propositional formulas obtained by selecting exactly one subformula for each choice operator.
Flexibility: This allows the representation of underspecified formulas. By adjusting the presence of choice operators, LoH can represent anything from a fully specified rule (no choices) to a completely data-driven search space (all choices).

2. Compilation to Differentiable Graphs

To enable learning via backpropagation, LoH formulas are compiled into differentiable computational graphs:

Weighted Gates: Each candidate subformula $\Phi_i$ in a choice operator is assigned a learnable weight $w_i \in [0, 1]$ .
Dual Compilation: The choice operator is converted into a propositional formula using either Disjunctive Compilation ( $\bigvee w_i \land \Phi_i$ ) or Conjunctive Compilation ( $\bigwedge \neg w_i \lor \Phi_i$ ). The choice between these depends on the operator's context (e.g., inside a conjunction vs. a disjunction) to optimize gradient flow.
Fuzzy Logic Semantics: The compiled formulas are interpreted using Gödel fuzzy logic (where conjunction is $\min$ and disjunction is $\max$ ). This allows the logical operations to be differentiable.

3. The Gödel Trick and Discretization

A critical component is the mechanism for learning discrete decisions while maintaining differentiability:

Parameterization: Instead of learning weights $w_i$ directly, the model learns logits $z_i$ .
Stochastic Local Search: During the forward pass, Gumbel noise is added to the logits. The values are then shifted so that exactly one logit is positive, and a sigmoid function is applied to produce weights.
Theorem 1 (Lossless Extraction): Leveraging properties of Gödel logic, the authors prove that for any fuzzy interpretation, rounding the output values using a threshold of 0.5 yields a Boolean interpretation consistent with the continuous model. This ensures that discretizing the learned model incurs no loss in accuracy compared to the continuous version, a property not guaranteed by other fuzzy logics like Product or Łukasiewicz.

4. Architectural Integration

LoH can be stacked on top of neural networks (e.g., CNNs) to learn perception-to-symbol mappings end-to-end. The choice operators can also be used to enforce syntactic templates (e.g., fixed-width clauses) or to select subsets of a known knowledge base.

Key Contributions

Novel Language (LoH): A formalism extending propositional logic with a choice operator, enabling the compact representation of hypothesis spaces and flexible integration of partial prior knowledge.
Differentiable Compilation: A procedure to compile LoH formulas into computational graphs using Gödel fuzzy logic and the Gödel trick. This allows for the end-to-end learning of discrete logical rules without accuracy degradation upon binarization.
Unifying Framework: LoH subsumes existing NeSy paradigms. It can function as a pure rule inducer (zero knowledge), a knowledge injector (full knowledge), or a hybrid system handling intermediate scenarios (partial knowledge, template constraints).
Empirical Validation: The framework demonstrates strong performance on tabular data and perceptual tasks (Visual Tic-Tac-Toe), outperforming or matching state-of-the-art models like MLLP and DLN while providing interpretable, discrete rules.

Experimental Results

The authors evaluated LoH on two main fronts:

1. Tabular Classification (12 UCI Datasets)

Setup: LoH models were compared against Decision Trees (DT), Random Forests (RF), XGBoost, standard Neural Networks (NN), Differentiable Logic Networks (DLN), and Multi-Layer Logical Perceptrons (MLLP).
Findings:
- LoH achieved competitive F1 scores, often matching or exceeding the performance of standard NNs and MLLP.
- Unlike MLLP, where the extracted rules (CRS) often suffer a significant performance drop compared to the continuous model, LoH's discrete rules maintain the same accuracy as the continuous model due to the Gödel semantics.
- LoH consistently outperformed CRS and DLN (in its discrete form) on most datasets, particularly in multi-class settings with fewer classes.

2. Visual Tic-Tac-Toe

Setup: A perceptual task where the input is a $3 \times 3$ grid of MNIST digits (0=X, 1=O, 2=Blank) instead of symbolic inputs. The goal is to predict if 'X' wins.
Findings:
- LoH (specifically the DNF variant) achieved the highest performance, finding the 100% correct logical formula in 4 out of 30 runs.
- Other models (MLLP, DLN) struggled with the symbolic evaluation after training, often failing to produce accurate discrete rules despite reasonable continuous performance.
- The model successfully learned to ground symbols (identifying X, O, and Blank from images) and discover the underlying logical structure simultaneously.

3. Wildfire Risk Simulation (Knowledge Regimes)

Setup: A synthetic task testing different levels of prior knowledge: Full Knowledge, Selecting Reliable Rules, Selecting One Rule per Set, and Partial Knowledge.
Findings: LoH successfully adapted to all regimes. When compared to a modified $\partial$ ILP baseline, LoH outperformed it in regimes involving large hypothesis spaces, demonstrating superior efficiency in handling complex rule selection and partial knowledge integration.

Significance and Claims

The paper claims that LoH provides a single, principled learning framework that bridges the gap between knowledge injection and rule induction. Its primary significance lies in:

Flexibility: It allows researchers to specify the exact amount of prior knowledge available, from "zero" to "full," and handle intermediate cases where rules need revision or completion.
Lossless Discretization: By utilizing Gödel logic and the Gödel trick, LoH guarantees that the discrete, interpretable rules extracted from the model are as accurate as the continuous model used during training. This addresses a major limitation in previous neurosymbolic approaches.
End-to-End Learning: It enables the simultaneous learning of symbolic rules and perception-to-symbol mappings, grounding abstract concepts in raw data without requiring pre-trained symbolic components.

The authors acknowledge limitations, noting that the current implementation is propositional and that scalability remains a challenge for first-order extensions due to the combinatorial explosion of groundings. However, they posit that LoH offers a robust foundation for future developments in neurosymbolic AI.