🤖 machine learning

ActivationReasoning: Logical Reasoning in Latent Activation Spaces

The paper introduces ActivationReasoning, a framework that embeds explicit logical reasoning into the latent activation spaces of large language models by mapping sparse autoencoder features to logical propositions, thereby enabling transparent, controllable, and robust multi-hop reasoning across diverse tasks and model architectures.

Original authors: Lukas Helff, Ruben Härle, Wolfgang Stammer, Felix Friedrich, Manuel Brack, Antonia Wüst, Hikaru Shindo, Patrick Schramowski, Kristian Kersting

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Lukas Helff, Ruben Härle, Wolfgang Stammer, Felix Friedrich, Manuel Brack, Antonia Wüst, Hikaru Shindo, Patrick Schramowski, Kristian Kersting

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a Large Language Model (LLM) like a massive, incredibly talented but slightly chaotic orchestra. It can play beautiful music (generate fluent text), but if you ask the conductor (the model) why it played a specific note, it can't really explain itself. The notes are all mixed together in a giant, tangled web of sound waves. This is what researchers call "superposition"—many different ideas are encoded in the same overlapping space, making it hard to see the individual instruments or control the music.

The paper introduces a new framework called Activation Reasoning (AR) to fix this. Think of AR as installing a smart conductor's podium that can listen to the orchestra's internal vibrations, identify specific instruments, and give them a logical script to follow.

Here is how it works, broken down into three simple steps:

1. Finding the Instruments (Finding Latent Representations)

First, the team uses a tool called a Sparse Autoencoder (SAE). You can think of the SAE as a high-tech "frequency analyzer." It listens to the orchestra's chaotic noise and separates it into distinct, clear notes.

The Goal: To find "monosemantic" features. This means finding a specific vibration in the model that only means "Bridge" or only means "San Francisco," rather than a messy mix of both.
The Result: They create a dictionary of these clear concepts. Some concepts are just one note (Single-feature), some are a chord of a few notes (Multi-feature), and some are complex rules like "If you hear a sad violin AND a minor key, it's 'Sadness'" (Relational-feature).

2. Turning Notes into Sentences (Activating Propositions)

Now, imagine the model is reading a sentence: "The Golden Gate Bridge is in San Francisco."

As the model reads, the SAE lights up. It detects the "Bridge" note, the "San Francisco" note, and the "USA" note.
Instead of just letting these notes float around, AR grabs them and turns them into logical propositions (simple statements like "Bridge is active" or "San Francisco is active").
It builds a scoreboard (an activation matrix) showing exactly which concepts are currently "on" and how loud they are.

3. The Logic Engine (Logical Reasoning)

This is the magic step. The team gives the system a set of logical rules, like a recipe book for thinking.

The Rule: "IF (Bridge) AND (San Francisco) AND (USA) are all active, THEN (Golden Gate Bridge) is true."
The Action: Even if the model never saw the phrase "Golden Gate Bridge" before, the logic engine sees the three ingredients (Bridge, SF, USA) on the scoreboard and deduces the new concept. It creates a new, higher-level idea from the raw ingredients.
The Control: Because the system now knows "Golden Gate Bridge" is logically true, it can steer the model to answer questions correctly or block unsafe answers. For example, if the model tries to say "The Golden Gate Bridge is in China," the logic engine sees the "USA" rule is violated and corrects the model's path.

Why is this a big deal? (The Results)

The paper tested this "smart conductor" on several difficult tasks where normal models usually get confused:

Multi-Step Logic (PrOntoQA): Imagine a riddle that requires five steps of deduction. Normal models get lost after step two. AR kept its cool, solving 93%+ of these puzzles regardless of how long the chain of logic was. It didn't get tired or confused.
Reading Between the Lines (Rail2Country): Sometimes people don't say "Red"; they say "The color of a tomato." Normal models often miss this connection. AR, however, understood that "Tomato" implies "Red" and used that to solve the puzzle. It could handle metaphors and similes that confused other models.
Real-World Safety (BeaverTails): The system was tested on tricky safety questions. It could distinguish between a "police officer with a gun" (safe in context) and a "criminal with a gun" (unsafe) by looking at the logical combination of concepts, rather than just reacting to the word "gun."

The Bottom Line

The paper claims that Activation Reasoning turns the messy, invisible brain of an AI into a structured, logical workspace.

Transparency: We can now see exactly which concepts the AI is using to make a decision.
Reliability: It doesn't get confused by complex riddles or tricky wording.
Control: We can give the AI a set of logical rules, and it will follow them, making the AI safer and more predictable.

In short, AR takes the AI's "gut feeling" (latent activations) and gives it a "logical brain" to back it up, making the AI not just a fluent talker, but a reliable thinker.

Technical Summary: ACTIVATIONREASONING (AR)

Problem Statement

Large Language Models (LLMs) excel at generating fluent text but suffer from opaque internal reasoning mechanisms. Their internal activations are distributed and entangled (a phenomenon known as superposition), encoding multiple unrelated features in overlapping dimensions. This obscures conceptual representations, limits interpretability, and hinders systematic reasoning, compositional generalization, and rule enforcement. While Sparse Autoencoders (SAEs) have emerged as a tool to surface latent features that often align with human-interpretable concepts, these features remain fragile, passive, and often polysemous or contextually unstable. Crucially, SAEs lack mechanisms for composition and higher-order reasoning. Conversely, formal logic offers explicit compositionality and transparency but presupposes discrete propositional units, which conflicts with the continuous, superposed representations of LLMs. This mismatch has historically hindered the direct integration of symbolic reasoning into neural architectures.

Methodology: ACTIVATIONREASONING (AR)

The authors propose ACTIVATIONREASONING (AR), a framework that embeds explicit logical reasoning directly into the latent activation space of LLMs. AR treats SAE-derived features as propositional units and composes them through user-defined rules. The framework operates in three distinct stages:

1. Finding Latent Representations

The first stage constructs a concept dictionary ( $D$ ) grounded in the LLM's latent space.

Formalization: A concept $c$ is defined as a tuple $(n_c, r_c, \tau_c)$ , where $n_c$ is a semantic identifier, $r_c$ is the latent representation function, and $\tau_c$ is a soft activation threshold.
SAE Integration: To address superposition, raw LLM activations are projected into a sparse feature space using SAEs to promote monosemanticity.
Representation Types: AR supports three forms of latent representation to handle varying levels of abstraction and polysemy:
- Single-feature ( $R_{single}$ ): Ties a concept to one SAE feature.
- Multi-feature ( $R_{multi}$ ): Associates a concept with a set of $k$ SAE features, aggregating evidence via weighted sums to handle fragmented or polysemous features.
- Relational-feature ( $R_{relation}$ ): Defines concepts via structured relations (e.g., decision trees) among SAE features, allowing for complex interactions like conjunctions or context-dependent exclusions (e.g., distinguishing hate speech from educational usage).
Extraction: Representations are induced automatically by identifying SAE features that best differentiate tokens labeled with a concept from those that are not, or manually assigned by experts.

2. Activating Propositions

During inference, AR monitors token-level activations to map them to atomic propositions.

Activation Scoring: For each token $t$ and concept $c$ , an activation score $a(c, t)$ is computed based on the chosen representation type (e.g., summing weighted feature values for $R_{multi}$ ).
Proposition Formation: Scores are thresholded against $\tau_c$ $τ_{c}$ to form an activation matrix $A$ $A$ . This matrix includes:
- Local Activations ( $A_{local}$ ): Token-level evidence.
- Global Activations ( $A_{global}$ ): Sequence-level aggregated evidence, capturing abstract semantics (e.g., sarcasm) that emerge only over a span of text.

3. Logical Reasoning

User-defined logical rules ( $L$ ) are applied to the activation matrix $A$ to infer higher-order structures.

Forward Chaining: The system iteratively applies rules (e.g., $Bridge \land San Francisco \land USA \rightarrow Golden Gate Bridge$ ) to the discretized activations.
Enriched Matrix: The process yields an enriched matrix $A'$ containing both directly detected concepts and inferred propositions. This allows the system to compose new concepts absent from the original SAE dictionary and perform multi-hop deduction.

Integration and Control

The enriched matrix $A'$ serves two purposes:

Analysis: Fine-grained inspection of model behavior, tracing failures to specific activations, and evaluating adherence to safety rules.
Control: Direct intervention during inference. AR can steer the model by amplifying or suppressing latent activations corresponding to specific concepts (e.g., steering towards "Safe" or "Red") using decoder weights derived from the SAE.

Key Contributions

Framework: Introduction of ACTIVATIONREASONING (AR), a method that grounds logical propositions in latent SAE features and enables compositional reasoning via explicit rules.
Benchmark: Proposal of Rail2Country, a novel benchmark for reasoning over meta-level concept descriptions (e.g., similes like "colored like a tomato" instead of explicit "red").
Empirical Validation: Demonstration that latent activations serve as a viable substrate for structured logical reasoning, enhancing transparency, robustness, and control across reasoning, abstraction, and safety tasks.

Experimental Results

The authors evaluated AR on Llama-3.1-8B and Gemma-2-9B backbones across four settings:

Multi-Hop Reasoning (PrOntoQA): AR solved deductive tasks with 1, 3, and 5 hops with accuracies exceeding 93%. Unlike baselines (including larger instruction-tuned models and reasoning-specialized models like DeepSeek-R1), AR's performance did not degrade as task complexity increased.
Meta-Level Abstraction (Rail2Country): In the R2C-Mono setting (explicit colors), AR significantly outperformed baselines. In the R2C-Meta setting (colors described via similes), vanilla SAEs collapsed to 0% accuracy, and base LLMs dropped to ~30%. AR maintained robust performance, achieving 86% on Gemma-2-9B, successfully bridging the gap between explicit lexical cues and implicit semantic descriptions.
Natural Language Reasoning (ProverQA): AR demonstrated robustness to linguistic diversity, achieving ~70% accuracy on "Hard" tier problems where all other models (including GPT-4o and DeepSeek-R1) fell below 50%.
Safety and Alignment (BeaverTails): AR improved balanced accuracy across 14 safety dimensions. The relational representation ( $R_{relation}$ ) achieved the best overall performance (83.0%), significantly outperforming base SAEs (57.9%) and single/multi-feature variants, particularly on abstract and context-sensitive safety concepts.

Significance and Claims

The paper claims that grounding logical structure in latent activations provides a model-agnostic mechanism that improves reasoning performance while simultaneously increasing transparency and control. Key claims include:

Robustness: AR scales robustly with reasoning complexity and generalizes to abstract, context-sensitive tasks where standard LLMs and even larger models fail.
Transferability: The approach transfers across different model backbones (Llama and Gemma).
Efficiency: AR incurs minimal runtime overhead compared to Chain-of-Thought (CoT) or reasoning-specialized models, as reasoning occurs in the activation space rather than through extensive token generation.
Auditable Control: By operating on discrete propositions derived from latent features, AR enables auditable mechanisms for safety and alignment, allowing for direct steering of model behavior without retraining.

The authors position AR as a step toward unifying the semantic richness of LLMs with the systematic generalization of logic, moving toward models that are more reliable, steerable, and aligned with human values. They acknowledge limitations regarding the dependency on SAE quality and the current reliance on manually defined rules, suggesting future work in automated rule induction and probabilistic reasoning.