💻 computer science

SliceGraph: Mapping Process Isomers in Multi-Run Chain-of-Thought Reasoning

The paper introduces SliceGraph, a post-hoc graph-based framework that reveals "process isomers"—distinct, structurally coherent reasoning trajectories that converge on the same correct answer—demonstrating that standard final-answer aggregation overlooks the rich, multi-route geometric structure of multi-run chain-of-thought reasoning.

Original authors: Kang Chen, Junjie Nian, Yixin Cao, Yugang Jiang

Published 2026-05-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kang Chen, Junjie Nian, Yixin Cao, Yugang Jiang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you ask a group of 64 different AI "thinkers" to solve the same math problem. They all write out their step-by-step reasoning (Chain-of-Thought) and, in the end, they all arrive at the exact same correct answer.

Traditionally, researchers would just look at the final answer and say, "Great, they all got it right." They would treat the 64 attempts as 64 independent, random guesses that happened to match.

SliceGraph argues that this is like looking at 64 people who all arrived at the same destination in a city and assuming they all took the exact same bus route. In reality, some might have walked through a park, others might have taken a subway, and some might have driven via a highway. They all ended up at the same spot, but their journeys were completely different.

Here is how the paper breaks this down using simple analogies:

1. The Map: SliceGraph

Instead of reading the text of the AI's thoughts (which can be misleading, like two people saying "I went left" but meaning different things), the authors built a map based on the AI's internal wiring.

The Analogy: Imagine the AI's brain is a massive city with millions of light switches (neurons). Every time the AI thinks a word, a specific set of switches lights up.
The Method: The researchers didn't look at the words; they looked at which switches were on. They grouped the thinking process into small "slices" (chunks of 256 words) and compared the "switch patterns" of these slices.
The Result: They built a graph (a map) where nodes are these thinking slices and lines connect slices that use similar internal switches. This map reveals the hidden "roads" the AI took.

2. The Discovery: Process Isomers

The paper introduces a new concept called Process Isomers.

The Analogy: In chemistry, "isomers" are molecules that have the exact same atoms and formula but are arranged differently in space (like a straight chain vs. a ring).
The Finding: The researchers found that for the same math problem, the AI often finds the correct answer through completely different internal routes.
- Route A: The AI might start by trying to guess the answer, realize it's wrong, and then switch to a logical proof.
- Route B: Another AI might start with the logical proof immediately, skip the guessing, and arrive at the same answer.
- The Isomer: Even though they are both "correct" and have the same final answer, they are "Process Isomers" because their internal journey (the arrangement of their thinking steps) is structurally different.

The Stat: In about 85% of the problems they tested, the correct answers weren't just one single path. Instead, the AI found the answer through multiple distinct "families" of routes. On average, if you pick two correct attempts, there's a 76% chance they took completely different internal roads to get there.

3. The Terrain: High-Value Cores

The researchers also mapped out where the "success" happens in this mental landscape.

The Analogy: Imagine the map has "high-value cores"—these are like sunny, fertile valleys where the AI is most likely to be correct.
The Finding: These fertile valleys are often disconnected.
- One "family" of routes might stay entirely in the "North Valley."
- Another "family" might stay entirely in the "South Valley."
- They both reach the finish line, but they never cross paths in the middle. They are specialized for different parts of the terrain.

4. Why This Matters (According to the Paper)

The paper claims that by only looking at the final answer, we are "collapsing" a rich, multi-dimensional world into a flat, boring line.

The Old Way: "They got the answer right. Good job." (Ignores how they got there).
The SliceGraph Way: "They got the answer right, but they used three different internal strategies, and those strategies never actually met in the middle."

The authors validated this by having human experts look at the slices. The experts confirmed that slices grouped together by the AI's internal switches did actually represent the same logical step or strategy, proving the map is accurate.

Summary

The paper says: Don't just trust the final answer. Even when an AI gets the right answer, it might be using a completely different "internal map" than another AI (or even the same AI on a different try). These different paths are called Process Isomers, and they reveal that the AI's reasoning is much more diverse and structured than we previously thought.

What the paper does NOT claim:

It does not say this makes the AI smarter or more reliable in the real world.
It does not suggest using this to fix AI errors or build new medical tools.
It strictly focuses on measuring and mapping how the AI thinks, not on changing how it thinks.

Technical Summary: SliceGraph: Mapping Process Isomers in Multi-Run Chain-of-Thought Reasoning

Problem Statement

Current approaches to multi-run Chain-of-Thought (CoT) reasoning, such as self-consistency, typically aggregate $N$ sampled trajectories solely by their final answers. This aggregation discards the internal structure of how these trajectories share, split, and rejoin through intermediate computation. Existing internal analyses often remain "within-run," studying semantic convergence or step-specific geometry within a single trajectory, rather than examining the joint structure across multiple runs for the same problem. Furthermore, prior graph-based pipelines often rely on intermediate projections (e.g., PCA, UMAP, sentence embeddings) to compute slice similarities. The authors argue that these projections distort the route geometry, either compressing distinct family counts or collapsing the entire atlas due to topical saturation, thereby obscuring the true multi-route structure of reasoning.

The core question addressed is: When multiple CoTs solve the same problem correctly, do they converge to a single process route, or do they form distinct "process isomers"—trajectories that share the same final answer but traverse different intermediate routes?

Methodology: SliceGraph

The authors propose SliceGraph, a post-hoc measurement object (not a decoding program) that maps the process geometry of sampled CoTs. The methodology consists of five layers:

1. SliceGraph Construction (The Scaffold)

Input: For a given problem-model cell, $N=64$ CoT runs are sampled.
Slice Definition: Activations are aggregated into 32-token rows, then further aggregated into coarser reasoning slices (canonical size: 256 tokens, 8 rows).
Representation: Instead of text embeddings or PCA, the method uses sparse activation keys. Each slice is represented as a binary set of the top- $k$ active MLP neurons (identified by layer-unit pairs).
Graph Construction: Nodes represent slices. Edges are established via mutual- $k$ NN based on Jaccard similarity of the sparse activation key sets. This preserves the unprojected, high-dimensional structure of the reasoning process.

2. Structural Decomposition

Biconnected Components (BCCs): The graph is decomposed into BCCs (maximal subgraphs remaining connected after removing any single node). These represent shared reasoning states.
Articulation Points: Nodes connecting BCCs act as branch points where the reasoning population forks.
Role Assignment: Blocks (BCCs) are assigned roles (e.g., shared_trunk, answer_basin, decision_point) based on run coverage, size, answer purity, and position.
Validation: Two blinded annotation studies confirm that BCCs correspond to human notions of shared reasoning states and that the resulting clusters align with shared high-level strategies.

3. Process Families and Isomers

Clustering: Runs are clustered into Process Families based on the set of non-trivial blocks they visit, using a weighted Jaccard similarity and Louvain community detection.
Definition of Process Isomer: Two correct CoT runs are defined as process isomers if they reach the same normalized final answer but belong to different process families. This indicates they used different intermediate routes to reach the same solution.

4. Label-Seeded Reward Field

To analyze the value landscape, a reward field is diffused over the fixed graph structure.
Seeding: A seed value is calculated for each block based on the excess success rate of runs visiting that block.
Diffusion: This seed is propagated via a PageRank-style diffusion process.
High-Value Cores: The top quartile of positive field values defines "high-value cores." If a cell has multiple disconnected high-value cores, it is "multi-core."

5. Typed-State Dynamics

Runs are lifted to a typed-state transition model where states are defined by (block role, time bin, core membership).
Family Kernels: Transition kernels are estimated for each process family.
Analysis: The authors compare these kernels using Total Variation (TV) distance against null controls (label shuffling) to determine if families navigate the atlas differently.

Key Results

1. Structural Validity and Representation

Semantic Alignment: Human annotators confirmed that slices within the same BCC represent the same reasoning operation (88.9–90.3% agreement) and that runs within the same process family use the same high-level strategy (94.4% agreement).
Scaffold Robustness: The unprojected sparse activation-key Jaccard similarity is critical. PCA and sentence embeddings (MiniLM) collapsed the multi-route geometry, reducing valid cells to near zero or compressing family counts significantly.

2. Prevalence of Process Isomers

Multi-Family Prevalence: In 85.5% of the 954 problem-model cells (across three 4B/8B models on math/science benchmarks), correct CoTs split into multiple process families.
Isomer Rate: Among normalized-correct run pairs, 76.6% are process isomers (belonging to different families).
Stability: This phenomenon persists under split-half resampling and holds even when controlling for the number of correct runs (e.g., at $m=4$ correct runs, the isomer rate remains 58.2%).

3. Value Landscape and Navigation

Multi-Basin Structure: 67.6% of reward-evaluable cells exhibit multiple disconnected high-value cores.
Specialization: Process families do not merely duplicate each other; they specialize over distinct high-value core footprints. The specialization score (measuring how concentrated a family's footprint is on specific cores) averages 0.59.
Distinct Dynamics: Family-specific transition kernels differ significantly. In 80.9% of multi-family cells, the transition dynamics between families exceed the null distribution (label shuffling), indicating that families navigate the same atlas using distinct transition kernels, not just by visiting different static states.

4. Cross-Architecture and Scale Replication

Replication: The findings hold across different architectures (DeepSeek-R1-Distill-Llama-8B) and scales (Qwen3-32B).
Scaling Effects: On the larger Qwen2.5-Math-72B model, the multi-family rate attenuates (67.3%) due to higher accuracy saturation (more deterministic solving), but the qualitative geometry (specialization, distinct dynamics) remains when structural diversity exists.

Significance and Claims

The paper claims that final-answer aggregation overlooks a structured multi-route process geometry. By treating the reasoning process as a graph of activation overlaps rather than a bag of independent samples, SliceGraph reveals that:

Process Isomerism is Common: Correct solutions to the same problem often follow fundamentally different intermediate routes (process isomers).
Route Families are Distinct: These isomers are not random variations but form coherent "process families" that specialize over different regions of the value landscape (high-value cores).
Navigation Dynamics Differ: These families do not just visit different states; they traverse the reasoning atlas with distinct transition kernels.

The authors position SliceGraph as a measurement object for auditing process diversity, collapse, and over-concentration, rather than a tool for driving decoding. The work suggests that understanding the "shape" of reasoning requires analyzing the joint structure of multiple runs via activation-key similarity, avoiding the distortions introduced by projection-based embeddings.