🤖 machine learning

AlgoTrace: Algorithmic Primitives and Compositional Geometry of Reasoning in Language Models

The paper introduces AlgoTrace, a framework that identifies and manipulates reusable algorithmic primitives within the latent space of large language models, demonstrating that multi-step reasoning can be steered and composed through geometric operations on these vectors to enhance cross-task generalization.

Original authors: Samuel Lippl, Thomas McGee, Kimberly Lopez, Ziwen Pan, Pierce Zhang, Salma Ziadi, Oliver Eberle, Ida Momennejad

Published 2026-08-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Samuel Lippl, Thomas McGee, Kimberly Lopez, Ziwen Pan, Pierce Zhang, Salma Ziadi, Oliver Eberle, Ida Momennejad

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Large language models are the engines behind modern artificial intelligence, capable of writing stories, solving equations, and navigating complex logic puzzles. For years, scientists have wondered how these systems actually think. Do they simply memorize patterns from their training data, or do they perform genuine, step-by-step reasoning? To answer this, researchers have begun looking inside the models, not at the words they produce, but at the invisible mathematical signals that flow through their internal layers as they work. These signals, often called latent activations, are the raw material of thought within the machine. The question has been whether these signals form a structured map where specific types of thinking happen in specific places, much like how different parts of the human brain specialize in different tasks. If such a map exists, it would mean that reasoning is not a chaotic blur, but a journey through a landscape of distinct, reusable mental tools.

A team of researchers has now mapped this landscape, revealing that large language models do indeed rely on a set of basic, reusable building blocks to solve problems. They call these building blocks "algorithmic primitives." Think of a primitive as a single, fundamental step in a larger process, such as finding the closest city to your current location when planning a trip, or checking if a mathematical equation balances. The researchers developed a new method to trace these steps as they happen inside the model. By watching how the model's internal signals change while it solves difficult problems, they were able to group similar moments of thought together. They found that when the model is solving a complex puzzle, it moves through a sequence of these distinct mental states, much like a traveler moving from one town to the next on a map.

To understand what was happening, the team tested the models on four very different types of challenges: finding the shortest route for a traveling salesman visiting multiple cities, solving logic puzzles with true and false statements, tackling difficult math competition problems, and navigating through a virtual maze. As the models worked through these tasks, the researchers recorded the internal signals at every step. They used a computer program to group these signals into clusters, finding that certain groups of signals consistently appeared when the model was doing specific things, like calculating a distance or verifying a solution. They then used an automated system to read the text the model was writing at those exact moments, confirming that the internal signals matched the external actions. For instance, one cluster of signals always appeared when the model was picking the nearest city to visit next, while another appeared when it was double-checking its work.

The most significant discovery was that these mental building blocks are not just passive markers; they can be actively controlled. The researchers extracted a specific "vector," which is essentially a direction in the model's internal space, for each of these primitives. They then injected these vectors back into the model while it was thinking. When they added the vector for "finding the nearest neighbor," the model immediately started using that strategy more often, even if it wasn't the best approach for the specific problem. When they added the vector for "generating a new path," the model began exploring more options. This proved that these internal signals are the direct cause of the model's behavior. By turning a specific mental switch on or off, the researchers could steer the model to think differently, suggesting that the model's reasoning is composed of these distinct, manipulable parts.

The study also showed that these building blocks can be combined. Just as you can mix primary colors to create new shades, the researchers found that adding two different primitive vectors together created a new, combined behavior. For example, they could take a vector that helped a model find the end of a path and another that helped it compare values, mix them, and the model would suddenly become capable of a more complex task that required both skills. This suggests that the model's ability to reason is not a single, monolithic skill, but a flexible geometry where simple operations can be assembled into complex solutions.

Furthermore, the researchers compared a standard model with one that had been specially trained to reason better. They found that the trained model used these building blocks more effectively. It relied less on brute-force guessing and more on structured steps like verification and planning. The trained model also showed a greater ability to transfer these skills from one type of problem to another. A mental tool learned while solving a math problem could be successfully used to help solve a navigation puzzle. This indicates that the training process did not just teach the model more facts, but actually strengthened its ability to compose and reuse these fundamental reasoning steps across different domains.

The findings offer a new way to look at artificial intelligence. Instead of viewing these models as black boxes that produce answers by magic, we can now see them as systems that walk through a structured space of algorithmic steps. The researchers demonstrated that by understanding the geometry of these steps, we can not only explain why a model makes a mistake but also fix it by adjusting the internal signals. This work suggests that the path to more reliable and capable artificial intelligence may lie in teaching these systems to better compose and generalize these basic building blocks, allowing them to solve problems they have never seen before by combining the tools they already possess.

Technical Summary: AlgoTrace: Algorithmic Primitives and Compositional Geometry of Reasoning in Language Models

Problem Statement

While inference-time compute and reasoning-finetuned models (e.g., Phi-4-Reasoning) have significantly improved the performance of Large Language Models (LLMs) on multi-step reasoning tasks, the internal mechanisms enabling this capability remain poorly understood. Specifically, it is unclear whether LLMs acquire generalized representations of algorithmic primitives (minimal computational operations like "retrieving the nearest neighbor" or "verifying a path"), whether these primitives are geometrically organized in the latent activation space, and whether complex reasoning arises from the composition of these primitives. Furthermore, the extent to which reasoning finetuning enhances the composition and generalization of these primitives across different domains is not well characterized.

Methodology: The AlgoTrace Framework

The authors introduce AlgoTrace, a framework designed to trace, extract, and steer algorithmic operations within the model's latent space. The methodology proceeds through four primary stages:

Latent Clustering and Annotation:
- The authors cluster internal representations (residual stream activations) of models (Phi-4, Phi-4-Reasoning, Llama-3, Qwen) while they solve multi-step reasoning tasks: Traveling Salesperson Problem (TSP), 3-SAT, AIME (Math Olympiad), and Graph Navigation.
- Using $k$ -means clustering (typically $k=50$ ) on layer 17 activations, they identify distinct clusters.
- An automated LLM-driven pipeline annotates these clusters by analyzing the tokens associated with them and their surrounding context, mapping latent clusters to specific algorithmic roles (e.g., "nearest neighbor selection," "path generation," "verification").
- A "cluster expressivity" metric (unique tokens divided by cluster radius) is used to filter for high-quality primitives.
Primitive Vector Extraction:
- Following the function vector approach (Todd et al., 2024), the authors extract primitive vectors from the attention heads of the identified clusters.
- These vectors represent directions in the activation space that, when injected into the residual stream, reliably induce a specific algorithmic primitive.
Primitive Validation and Steering:
- Injection: Primitive vectors are injected into the residual stream at various layers and magnitudes during inference.
- Behavioral Hallmarks: The authors define specific behavioral metrics (e.g., proportion of nearest-neighbor paths, number of unique paths generated, timing of distance computations, frequency of verification words) to quantify the induction of the target primitive.
- Causal Patching: They verify that injecting a vector increases the associated behavior, while subtracting it decreases the behavior.
Compositional Geometry and Transfer:
- Algebraic Composition: The authors test if primitive vectors can be combined via addition or subtraction in activation space to induce composite behaviors (e.g., adding a "Terminal Node Recognition" vector and a "Reward Comparison" vector to solve a composite task).
- Cross-Task Transfer: They test whether vectors extracted from one domain (e.g., AIME) can steer behavior in another (e.g., TSP).
- Model Comparison: They compare base models (Phi-4) against reasoning-finetuned models (Phi-4-Reasoning) to analyze differences in primitive usage and composition.

Key Results

Identifiability of Primitives: The framework successfully identified interpretable algorithmic primitives (e.g., "nearest neighbor," "compute distance," "compare/verify") that correspond to specific clusters in the latent space. These clusters showed high expressivity and consistent labeling across manual and automated annotation.
Effective Steering: Injecting primitive vectors selectively induced the associated algorithmic behavior. For example, injecting a "nearest neighbor" vector increased the proportion of nearest-neighbor paths generated by the model by +56.1%, while injecting a "compute distance" vector accelerated distance computations.
Compositional Geometry: The study demonstrated that primitive vectors exhibit geometric regularities. Adding primitive vectors (e.g., $v_{TNR} + v_{RC}$ ) successfully induced composite behaviors in models like Llama-3-8B, allowing them to solve composite tasks in a zero-shot setting with performance matching few-shot baselines.
Cross-Task Transfer: Primitive vectors extracted from one task (e.g., AIME) successfully transferred to and steered behavior in a different task (e.g., TSP). Injecting AIME-derived "spatial reasoning" and "plan final answer" vectors into Phi-4 solving TSP increased the generation of paths and nearest-neighbor heuristics, mimicking the behavior of the reasoning-finetuned model.
Impact of Reasoning Finetuning:
- Stereotyping: Phi-4-Reasoning responses were highly stereotyped and consistently employed heuristic strategies (e.g., nearest-neighbor), whereas the base Phi-4 model exhibited distinct subgroups of responses (brute-force vs. random guessing).
- Enhanced Composition: The reasoning-finetuned model showed improved composition of primitives, particularly in verification and planning steps, suggesting that finetuning strengthens algorithmic generalization and the ability to compose primitives across domains.

Key Contributions

Systematic Identification: The first systematic identification of cross-domain algorithmic primitives in LLMs, linking explicit reasoning traces to internal latent representations.
Geometric Framework: A geometric framework demonstrating that primitive composition follows algebraic operations (addition, subtraction, scalar modulation) in activation space.
Methodological Link: A novel methodology connecting explicit reasoning behaviors to internal mechanisms via automated LLM annotation and causal vector injection.
Mechanistic Evidence: Evidence that reasoning finetuning improves the composition and generalization of algorithmic primitives, shifting models from memorization toward compositional problem-solving.
Evaluation of Transfer: Empirical demonstration of cross-task primitive transfer, showing that algorithmic building blocks are shared across diverse reasoning domains.

Significance and Claims

The paper claims that LLM reasoning can be understood as a "walk through algorithmic primitives in the latent space governed by compositional geometry." The findings suggest that:

Generalization: Reasoning-finetuned models do not merely memorize task-specific patterns but acquire generalized algorithmic capacities that can be composed and transferred.
Interpretability: The "algorithmic fingerprint" of a model (the frequency and composition of primitives) provides a granular view of its reasoning strategy, distinguishing between heuristic, brute-force, and random approaches.
Steering: Understanding these primitives allows for precise steering of model behavior, potentially enabling the correction of reasoning failures or the enhancement of specific algorithmic capabilities without retraining.

The authors position this work as a step toward a "principled path from laboratory-identified primitives to real-world reasoning behaviors," offering a framework for evaluating and potentially improving the algorithmic foundations of LLMs. They acknowledge limitations, such as the current focus on linear compositions and the fact that identified primitives do not always map perfectly to known algorithms, but argue that this framework provides a rigorous basis for future research into the "geometry of abstraction" in language models.