🤖 AI

Accelerating NeurASP with vectorization and caching

This paper enhances the scalability of the neurosymbolic framework NeurASP by implementing vectorization, batch processing, and caching to achieve multi-order-of-magnitude speedups in training, validated through a new challenging card-playing dataset.

Original authors: Alexander Philipp Rader, Alessandra Russo

Published 2026-06-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alexander Philipp Rader, Alessandra Russo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Teaching a Robot to Think

Imagine you are trying to teach a robot to play a card game. You have a Neural Network (the robot's "intuition") that looks at pictures of cards and guesses what they are. You also have a set of Logic Rules (the robot's "brain" or "rulebook") that takes those guesses and figures out the final score.

The problem is that you only know the final score (e.g., "The total is 15"). You don't know what the robot guessed for each individual card (e.g., "Is that a 7 of Hearts or a 3 of Spades?").

In the past, the framework called NeurASP tried to learn this by guessing every possible combination of cards that could result in a score of 15, checking which ones worked, and then adjusting the robot's intuition. However, this process was like trying to count every grain of sand on a beach one by one. It was so slow that for complex tasks, the computer would simply give up and time out before finishing.

This paper introduces a supercharged version of NeurASP that makes this process thousands of times faster, allowing the robot to learn complex card games that were previously impossible.

The Three Bottlenecks (and how they were fixed)

The authors identified three main reasons why the original system was so slow, and they fixed each one with a clever trick.

1. The "Calculator" Problem (Vectorization)

The Old Way: Imagine a teacher asking a student to multiply numbers. The old NeurASP code asked the student to do the math one number at a time, in a long line. "Multiply 2 by 3. Okay, now multiply that by 4. Okay, now multiply that by 5..." This is very slow.
The New Way: The authors taught the computer to use Vectorization. Instead of doing math one by one, they gave the computer a giant calculator that can multiply a whole stack of numbers all at once. It's like switching from a single-lane dirt road to a 10-lane highway.

Result: Calculating probabilities and gradients (the math needed to learn) became 100 to 1,000 times faster.

2. The "Library" Problem (Caching)

The Old Way: Imagine you are solving a puzzle. Every time you see a puzzle piece, you walk to a library, find the solution for that specific piece, write it down, and then walk back. Even if you see the same piece 1,000 times, you walk to the library 1,000 times.
The New Way: The authors realized that the solution to a puzzle piece depends only on the final score, not on the specific picture of the card. If the final score is "15," the possible card combinations are always the same, no matter what the cards look like.
They built a Cache (a personal notebook). The first time they see a score of "15," they walk to the library, find the answer, and write it in the notebook. The next 999 times they see a "15," they just look in the notebook.

Result: Instead of asking the library for answers 30,000 times, they only asked 19 times (once for every possible score). This saved massive amounts of time.

3. The "Delivery" Problem (Batching)

The Old Way: The original code was like a delivery driver who dropped off one package, went back to the warehouse, picked up the next package, and drove back out.
The New Way: The new code uses Batch Processing. It loads a whole truck full of packages (data) and delivers them all in one trip. This is much more efficient for large datasets.

The New Challenge: "Card Arithmetic"

To prove their new system works, the authors created a new, difficult test called Card Arithmetic.

The Old Tests: Previous tests used simple numbers (like adding two digits from 0-9). It was like adding 2 + 3.
The New Test: They used real photos of playing cards. There are 52 different cards (Ace of Spades, 2 of Hearts, etc.), and the rules involve multiplying and adding their values.
The Difficulty: For a single game, there could be tens of thousands of different ways the cards could be arranged to get the right total.

The Results

When they ran the experiments:

Speed: The new system finished tasks in minutes that the old system would have taken days, weeks, or months to complete. In some cases, the old system simply crashed (timed out) because it was too slow.
Accuracy: The new system successfully learned the card games, achieving high accuracy even with the massive complexity of 52 different cards.
Comparison: They compared their system to another framework called Embed2Sym. While Embed2Sym was good at simple tasks, it failed completely on the card games because it got stuck trying to solve the massive number of possibilities. NeurASP handled it easily.

A Quirk: "Reasoning Shortcuts"

The paper also noticed something interesting. Sometimes, the robot got the final score right but guessed the individual cards wrong.

Analogy: Imagine the rule is "3 of Clubs (value 6) + 2 of Spades (value 6) = 12." The robot might guess "6 of Hearts + 6 of Diamonds = 12." It got the math right, but the cards were wrong.
Because different cards can have the same mathematical value, the robot found a "shortcut" to get the right answer without learning the true identity of the cards. This shows that while the system is fast, it sometimes finds clever, but technically incorrect, ways to solve the puzzle.

Summary

The authors took a slow, clunky system for teaching AI to reason with logic and rules, and they turbocharged it. By organizing the math better (vectorization) and remembering past answers (caching), they turned a system that could barely handle simple addition into one that can master complex card games in a fraction of the time.

Technical Summary: Accelerating NeurASP with Vectorization and Caching

Problem Statement
Neurosymbolic AI frameworks like NeurASP combine neural networks for concept perception with Answer Set Programming (ASP) for symbolic reasoning. A critical limitation of the original NeurASP implementation is its computational inefficiency, which hinders scalability to complex tasks. The framework relies on semantic loss to backpropagate learning signals from downstream labels (which are available) to latent concepts (which are not). This process requires three expensive steps: computing all possible answer sets (latent combinations) that satisfy a downstream label, calculating the probability of each answer set based on neural confidences, and computing gradients for these probabilities.

Previous applications of NeurASP to more challenging datasets have frequently resulted in timeouts. The authors attribute these failures not to theoretical limits of the framework, but to an imperfect implementation characterized by naive nested loops for probability and gradient calculations and suboptimal caching strategies for answer set generation.

Methodology and Technical Improvements
The authors propose an enhanced implementation of NeurASP that addresses these bottlenecks through three primary technical modifications:

Vectorization: The original implementation used nested Python loops to calculate probabilities (Equation 1) and gradients (Equation 2) for answer sets. The new implementation replaces these loops with vectorized PyTorch tensor operations. By stacking neural confidences into 2D tensors and utilizing indexing and reduction functions (e.g., prod, sum), the system processes all interpretations and answer sets simultaneously rather than iteratively.
Answer Set Caching: The original code cached answer sets per dataset example, still requiring a call to the ASP solver (Clingo) for every training instance. The authors observed that answer sets depend solely on the downstream label, not the specific input data. Consequently, the new implementation caches answer sets per unique downstream label. For a dataset with 30,000 examples but only 19 unique labels (e.g., MNIST addition sums 0–18), this reduces Clingo calls from 30,000 to 19.
Infrastructure Enhancements: To support large-scale experimentation, the framework was refactored to include batch processing via data loaders, validation sets for hyperparameter tuning, random seeding for reproducibility, and extensive JSON logging. Unit tests were added to ensure functional correctness.

Experimental Results
The authors evaluated the improved framework using synthetic data and benchmark tasks, including standard MNIST-based tasks (Addition, Member) and a new, more complex dataset called "Card arithmetic" involving playing card images.

Synthetic Speed Tests: On synthetic data varying the number of answer sets (100 to 1,000,000), the improved NeurASP demonstrated speedups of two orders of magnitude for probability calculations and three orders of magnitude for gradient calculations compared to the original NeurASP. It also outperformed SLASH (a recent extension of NeurASP) by an average factor of 100x on gradient calculations due to SLASH's continued use of partial looping.
Benchmark Execution Times: On real datasets, the improvements were drastic. For the "Member 5" task (50,000 answer sets), the original NeurASP required over 4 hours, while the improved version completed in 12 minutes. For the most complex "Card sum 4" task (40,130 answer sets), the original NeurASP and SLASH were extrapolated to take days or weeks (or timed out), whereas the improved NeurASP completed training in approximately 18 hours.
Accuracy and Scalability: The improved NeurASP achieved high downstream and latent accuracies on Card arithmetic tasks (e.g., 86% latent accuracy on Card sum 4), successfully solving problems where the competing framework Embed2Sym timed out. Embed2Sym failed on Card tasks because its optimization step involves assigning 52 clusters to concepts, leading to a combinatorial explosion ( $52!$ solutions). While NeurASP struggled with "reasoning shortcuts" in the "Card prodsum" tasks (where different card combinations yield the same value, leading to high variance in latent accuracy despite correct downstream predictions), it remained the only framework capable of converging on these tasks within a reasonable timeframe.

Key Contributions

Algorithmic Optimization: The introduction of vectorized probability and gradient calculations and a label-based answer set caching mechanism, which collectively reduce computation time by multiple orders of magnitude.
New Dataset: The creation of the "Card arithmetic" dataset, featuring high-resolution playing card images and 52 latent concepts, designed to test the limits of neurosymbolic scalability beyond standard MNIST/CIFAR benchmarks.
Framework Modernization: A complete refactoring of the NeurASP codebase to support batch processing, reproducibility, and robust logging, making it viable for extensive experimentation.

Significance
The paper claims that these improvements transform NeurASP from a framework limited to simple perception tasks into a scalable tool capable of handling complex neurosymbolic problems with tens of thousands of answer sets and dozens of latent concepts. By reducing training times from months to hours, the work enables the application of NeurASP to more sophisticated real-world problems. Furthermore, the successful training on Card arithmetic tasks demonstrates that semantic loss can effectively guide learning in high-complexity latent spaces, although the authors note that the volatility of semantic loss and the emergence of reasoning shortcuts remain areas for future investigation. The authors conclude that their work paves the way for further research into the strengths and limits of NeurASP and semantic loss in neurosymbolic AI.