💻 computer science

Neurosymbolic Object-Centric Learning with Distant Supervision

The paper introduces DeepObjectLog, a probabilistic neurosymbolic model that learns object-centric representations directly from global task labels via distant supervision, enabling superior out-of-distribution generalization on visual reasoning tasks without requiring per-object annotations.

Original authors: Stefano Colamonaco, David Debot, Giuseppe Marra

Published 2026-05-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Stefano Colamonaco, David Debot, Giuseppe Marra

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to solve a math problem, but you only show it the final answer, not the steps or the individual numbers.

For example, you show the robot a picture with a "3" and a "4" scribbled on it, and you tell it, "The answer is 7." You don't tell the robot where the 3 is, where the 4 is, or even how many numbers are in the picture. You just give the picture and the final sum.

This is the challenge the paper tackles. Most AI models are like students who memorize the answer key but can't actually do the math if the numbers change. They struggle when the picture has a different number of items or a new combination of objects.

The Problem: The "Black Box" vs. The "Logic Puzzle"

Current AI falls into two camps, both with flaws:

Pure Neural Networks (The Black Box): These are great at recognizing patterns but terrible at reasoning. If you train them to add two numbers, they might learn to recognize the shape of the sum "7" rather than actually adding 3 + 4. If you show them 4 + 3, they might get confused because the pattern looks slightly different.
Neurosymbolic AI (The Logic Puzzle): These models use strict logical rules (like "If A + B = C"). They are great at reasoning, but they usually need a human to tell them exactly where the objects are first. They can't look at a messy photo and say, "Ah, there's a 3 over here and a 4 over there." They need the objects pre-sliced and handed to them.

The Solution: DeepObjectLog (The Detective)

The authors created a new system called DeepObjectLog. Think of it as a detective who is learning to solve a mystery by looking at the crime scene and the final verdict, without being told who the suspects are.

Here is how it works, using a simple analogy:

1. The "Slot" System (The Detective's Magnifying Glass)
Imagine the AI has a set of invisible "slots" or magnifying glasses it can place anywhere on an image. It doesn't know how many objects are there, so it spreads out, say, 5 magnifying glasses.

Some glasses land on a "3".
Some land on a "4".
Some land on empty background space.

2. The "Objectness" Question (Is it a suspect?)
For every magnifying glass, the AI asks: "Is this actually an object, or just background noise?"

If it's background, the AI says, "Ignore this one."
If it's an object, it says, "This is a suspect!" and tries to guess what it is (e.g., "This looks like a 3").

3. The "Logic Layer" (The Judge)
This is the magic part. The AI takes all its guesses ("Suspect A is a 3, Suspect B is a 4, Suspect C is nothing") and runs them through a strict logic program (like a math rulebook).

The rulebook says: "If you add the numbers of the real suspects, the result must match the final answer we were given (7)."
If the AI guessed wrong (e.g., it thought the background was a 5), the math won't add up to 7. The logic layer says, "That doesn't make sense," and sends a signal back to the detective to change its mind.

4. Learning from the Mistake
Over and over, the AI adjusts its magnifying glasses. It learns that when the final answer is 7, it must find a 3 and a 4. It learns to ignore the background noise because including it breaks the math.

Why This is a Big Deal

The paper claims this approach is superior because it learns structure rather than just memorization.

Generalization (The "New Puzzle" Test): If you train the AI on pictures with 2 or 3 numbers, and then show it a picture with 5 numbers, it can still solve it. Why? Because it learned the rule of addition, not just the pattern of "two numbers." It can activate more "magnifying glasses" to handle the extra numbers.
No Labels Needed: The AI didn't need a human to draw boxes around the numbers. It figured out where the objects were just by trying to satisfy the final answer.
Better than "Big Brain" Models: The authors tested their system against massive, pre-trained AI models (like the ones that chat with you). Even those huge models struggled with this specific logic puzzle, getting the answers wrong. DeepObjectLog, which is much smaller and specialized, got it right 90% of the time.

The Bottom Line

DeepObjectLog is like teaching a child to do math by showing them the answer key and letting them figure out the numbers themselves. By combining the ability to "see" objects (perception) with the ability to "check the math" (logic), the AI learns to understand the world in a way that is flexible, logical, and doesn't break when the situation changes.

The paper shows this works on tasks like adding numbers, recognizing poker hands, and counting objects in complex scenes, proving that AI can learn to "think" about objects even when it isn't told exactly what those objects are.

Technical Summary: Neurosymbolic Object-Centric Learning with Distant Supervision

Problem Statement

Current neurosymbolic AI systems face a critical bottleneck: while they can effectively use symbolic rules to supervise latent concepts from weak labels, they typically assume that the entities (objects) referenced by these rules are already specified. Conversely, object-centric learning models can decompose images into slot-like representations but often fail to align these slots with the specific predicates required for symbolic reasoning, especially when object-level supervision (e.g., masks, bounding boxes, or per-object labels) is unavailable.

This paper addresses the challenge of object-centric neurosymbolic learning under distant supervision. The core problem is defined as learning a model where:

Input: A dataset of image-label pairs $(x, y)$ where images may contain an unknown number of objects, but no annotations exist for object identity, location, or class.
Constraint: A probabilistic logic program $L$ is provided, specifying how latent object-level predicates (e.g., objectness and class) determine the global label $y$ .
Goal: To learn a neurosymbolic model that infers the object-centric structure (which parts of the image are objects and what they are) directly from the global task label, enabling the symbolic layer to constrain the perceptual layer without explicit object-level supervision.

Methodology: DeepObjectLog

The authors propose DeepObjectLog, a probabilistic neurosymbolic architecture that integrates a slot-based perceptual encoder with a probabilistic logic layer. The model is trained end-to-end using a Maximum A Posteriori (MAP) objective with amortized inference.

1. Probabilistic Graphical Model

The model defines a joint distribution over the image $x$ , the task label $y$ , and latent variables including scene representation $z$ , slot representations $s$ , objectness probabilities $\beta$ , objectness flags $o$ , and class labels $c$ .

Encoder: A Slot Attention mechanism extracts up to $N$ latent slot representations $s_i$ from the image.
Object Detector & Classifier: For each slot, the model predicts:
- An objectness probability $\beta_i \in [0, 1]$ , representing the uncertainty that the slot corresponds to a meaningful object.
- A class distribution $p(c_i | s_i, \beta_i)$ , conditioned on the objectness.
- The objectness flag $o_i$ is modeled as a Bernoulli variable with parameter $\beta_i$ .
Soft Masking: To reduce the influence of background or spurious slots, the slot representation is element-wise multiplied by its objectness score $\beta_i$ before being passed to the classifier and decoder.

2. Probabilistic Logic Reasoning

The symbolic component utilizes ProbLog, a probabilistic logic programming framework.

Probabilistic Facts: The objectness flags $o_i$ and class assignments $c_i$ are treated as probabilistic facts (e.g., $p::object(i)$).
Rules: Logical rules define how these facts combine to determine the task label $y$ (e.g., $add(Z) \leftarrow digit(1, Y1), digit(2, Y2), Z \text{ is } Y1 + Y2$ ).
Marginalization: The likelihood of the observed label $y$ is computed by marginalizing over all possible objectness and class assignments. This allows the system to compute a differentiable task-level learning signal without requiring discrete sampling or heuristic set-matching losses (like Hungarian loss).

3. Learning Objective

The training objective maximizes the joint probability of the observed data and a latent explanation $z$ :
$\max_{\theta, \phi} \sum_{(x,y) \in D} \log p_\theta(y | s^*(q_\phi(x)), \beta^*(q_\phi(x))) + \log p_\theta(x | s^*(q_\phi(x)), \beta^*(q_\phi(x))) + \log p(q_\phi(x))$
Where:

The first term encourages the inferred object-centric representation to explain the symbolic task label.
The second term (reconstruction loss) ensures the representation can reconstruct the input image.
The third term is a prior on the amortized latent explanation.
$q_\phi$ is a neural encoder that approximates the latent explanation $z$ , avoiding the need for an inner optimization loop per training example.

Key Contributions

Formulation: The paper formulates the problem of object-centric neurosymbolic learning from distant supervision, where objects must be inferred from perception rather than provided as structured inputs.
Probabilistic Interface: It introduces a differentiable interface between slots and logic where objectness and class variables are marginalized via probabilistic inference. This eliminates the need for object annotations, discrete sampling, or set-matching losses.
DeepObjectLog Architecture: The authors instantiate this formulation in DeepObjectLog and demonstrate that logical feedback improves the task-relevant use of object-centric representations, supporting stronger relational generalization.

Experimental Results

The model was evaluated on four benchmarks: MultiMNIST-Addition, PokerRules, CLEVR-Addition, and COCOLogic.

Compositional Generalization: On MultiMNIST-Addition, DeepObjectLog achieved 90.00% accuracy on out-of-distribution (OOD) compositions (unseen digit combinations), significantly outperforming neural baselines (e.g., Slot Attention at 13.30%) and a standard neurosymbolic baseline without object-centric perception (35.76%).
Task Generalization: In the PokerRules task, the model achieved 72.23% accuracy on OOD classes (e.g., "three of a kind" not seen in training), whereas the neurosymbolic baseline failed (0.46%). Neural baselines could not generalize to new classes as their output spaces were fixed by training labels.
Object Count Generalization: The model demonstrated robust extrapolation to images with more objects than seen during training (e.g., 4-5 digits in MultiMNIST), maintaining high accuracy where competitors degraded significantly.
Learning Signal Strength: On COCOLogic, DeepObjectLog achieved non-trivial concept accuracy with 0% object-level supervision, demonstrating its ability to learn object structure purely from the task signal. This performance was maintained even as supervision levels increased.
Comparison with VLMs: On the MultiMNIST-Addition OOD task, DeepObjectLog (90% accuracy) significantly outperformed large Vision-Language Models (LLaVA-1.5 and Qwen2.5-VL), which struggled to solve the task reliably under zero-shot prompting.

Significance and Claims

The paper claims that DeepObjectLog represents a significant step toward more general neurosymbolic systems by bridging the gap between perceptual object discovery and symbolic reasoning. Its primary significance lies in:

Relational Generalization: The ability to support reasoning under distribution shifts, including new object compositions, different object counts, and new symbolic queries over learned vocabularies.
Robustness without Supervision: The capability to learn meaningful object-centric structures and reason over them without requiring explicit object-level supervision (masks or bounding boxes), relying solely on global task labels and symbolic rules.
Interpretability: The model provides interpretable object-level predictions (objectness and class) that align with the symbolic reasoning process, offering guarantees that purely neural approaches lack.

The authors acknowledge limitations, including the computational cost of exact inference in probabilistic logic (though mitigated by optimization frameworks) and the potential for reasoning shortcuts in weakly supervised settings, which their structural constraints aim to discourage. They conclude that while the approach advances robust and interpretable AI, future deployments must consider risks related to bias and dual-use surveillance.