💬 NLP

J-CoT: Chain-of-Thought in J-Space

This paper introduces J-CoT, a recurrent reasoning framework that improves upon existing latent-reasoning methods by propagating intermediate states as vocabulary-indexed coefficients within a "J-space" coordinate system, thereby enabling efficient computation without the need for fully verbalized rationales or dense hidden state recurrence.

Original authors: Junde Wu, Jiayuan Zhu, Fengling Liu, Minhao Hu, Jiazhen Pan

Published 2026-07-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Junde Wu, Jiayuan Zhu, Fengling Liu, Minhao Hu, Jiazhen Pan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Secret Language of Thinking Machines

Imagine you are trying to teach a super-smart robot how to solve a tricky puzzle. In the past, the best way to do this was to force the robot to "think out loud." It would have to write down every single step of its reasoning in perfect English sentences before it could give you the final answer. This is called Chain-of-Thought, and it works well because it slows the robot down, letting it check its work. But there's a catch: forcing a robot to speak in full sentences is like asking a human to whisper their entire internal monologue to a friend before they can even finish a thought. It's slow, and it wastes energy on grammar and storytelling when the robot just needs to do some math or logic.

On the other end of the spectrum, some scientists tried to let the robot think in total silence, passing secret, invisible signals between steps. This is called Latent Reasoning. It's faster, but it's like trying to pass a note in class that is written in a code so dense and messy that no one knows exactly what information is actually important. The robot might get lost in a sea of numbers without a clear way to organize its thoughts.

So, the big question for researchers is: Is there a middle ground? Can we give the robot a way to pass its thoughts forward that is organized and clear, but doesn't waste time turning every tiny idea into a full sentence? This is the problem the paper "J-COT: Chain-of-Thought in J-Space" sets out to solve.

The Paper's Big Idea: The "J-Thought"

The authors introduce a new method called J-CoT (J-Chain-of-Thought). Instead of forcing the robot to write a story or pass a messy blob of data, J-CoT uses a clever middleman called a J-thought.

Think of the robot's brain as a massive library with millions of books. When the robot is thinking, it usually pulls out whole books (sentences) to read and write. But J-CoT suggests that between steps, the robot doesn't need to pull out the whole book. Instead, it just needs to pass a library card with a list of specific page numbers and highlights.

Here is how it works in their "J-space" system:

The Vocabulary Index: Imagine every word in the robot's vocabulary has a unique address in its brain.
The J-Thought: At the end of a thinking step, the robot doesn't write a sentence. Instead, it looks at its brain and says, "I'm focusing on the concepts related to these specific word-addresses, and here is how strong my focus is." It creates a short list of numbers (coefficients) attached to these word-addresses.
The Handoff: This list is the J-thought. It's passed to the next step. It's not a sentence, so it's fast. But because it's tied to specific words, it's organized and meaningful.
The Next Step: The robot takes this list, looks up the "addresses" in its brain, and uses that information to continue thinking. It can then do all its heavy lifting in its full, complex brain without being forced to speak until it's ready to give the final answer.

What They Found

The researchers tested this idea on some very hard tasks, like solving complex math problems, answering science questions, and writing computer code. They compared J-CoT against the old "think out loud" method and the "secret code" method.

The results were quite promising. They found that J-CoT-Zero (a version that didn't require any extra training) was already just as good as, or sometimes better than, the best "secret code" methods they tested. It managed to match or beat the top performers on every single benchmark they tried, from math puzzles to coding challenges.

Even better, when they trained a version called J-CoT-Train to get even better at reading and writing these "library cards," it became the clear winner. It scored the highest points across all the math, science, coding, and logic tasks they tested. For example, on a tough math test called MATH-500, the trained version jumped from a 50.4% score (the previous best) to 54.0%.

Why It Matters (And What It Isn't)

The paper suggests that this "J-space" interface is a sweet spot. It proves that you don't need to force a robot to speak in full sentences to make it think clearly, but you also don't need to let it wander in a fog of invisible numbers. By using this vocabulary-indexed list, the robot can keep its thoughts organized without the overhead of grammar.

However, the authors are careful not to claim this is a magic bullet that solves everything forever. They note that their results are based on specific tests and models. They also point out that while the "J-thought" is a great middle ground, the robot still needs to do its heavy computing inside its brain; the J-thought is just the messenger.

In short, the paper suggests that by changing how a robot passes its thoughts from one step to the next—from full sentences to a smart, word-based list—we can make it smarter, faster, and more efficient at solving hard problems. It's like realizing that to get a message across a crowded room, you don't need to shout a paragraph or whisper a secret code; you just need to hold up a sign with the right keywords.

Technical Summary: J-CoT: Chain-of-Thought in J-Space

1. Problem Statement

Large Language Models (LLMs) currently rely on two primary paradigms for recurrent reasoning, both of which present limitations:

Explicit Chain-of-Thought (CoT): This approach uses natural language as the recurrent interface. While effective, it forces every intermediate state to be serialized into discrete tokens. This requires the model to commit to grammatical forms, discourse continuity, and explicit explanations even when these are unnecessary for the next reasoning step. It constrains the state passed between cycles to a specific linguistic interpretation.
Dense Latent Reasoning: Methods like Coconut propagate continuous hidden states between computation steps, avoiding the need for verbalization. However, these methods pass a dense hidden vector as a whole. This interface lacks an explicit mechanism for selecting and organizing the specific information required by the next reasoning step, potentially carrying irrelevant or unstructured noise.

The paper posits a need for an intermediate interface that is linguistically grounded (anchored in the model's vocabulary) but non-sentential (not requiring a decoded sentence), allowing for full hidden-space computation within cycles while providing a structured state for recurrence.

2. Methodology: J-CoT

The authors introduce J-CoT, a recurrent reasoning framework operating within J-space, a vocabulary-indexed coordinate system derived from the model's hidden representations.

2.1 Core Concept: The J-Thought

Instead of passing decoded tokens or dense vectors, J-CoT passes a J-thought: a vocabulary-indexed coefficient state.

J-Space Construction: The method constructs a dictionary for each layer $\ell$ by pulling back final-layer vocabulary directions through the averaged downstream Jacobian. This creates a shared vocabulary index across different layer geometries.
Extraction and Reconstruction:
- Write (Extraction): At a designated write layer, the model extracts non-negative coefficients over the J-space dictionary from specific "carrier" positions using an elastic-net decomposition. This forms the J-thought matrix $A_t$ .
- Read (Reconstruction): At a designated read layer, these coefficients are reconstructed into the residual stream using the read-layer dictionary. A learned read gate regulates the integration of this state with the prompt-conditioned baseline.
Carrier Positions: The model appends $M$ non-linguistic carrier positions to the input. These positions participate in standard self-attention and feed-forward computations but are never decoded. They serve as the read/write locations for the J-thought.

2.2 Recurrent Mechanism

The J-CoT cycle operates as follows:

Initialization: The prompt and neutral carrier embeddings are propagated to the read layer.
Read: The previous J-thought is reconstructed at the read layer and gated into the carrier activations.
Compute: The carriers and prompt propagate through the intermediate Transformer layers (from read layer to write layer) using dense, unrestricted hidden representations.
Write: At the write layer, new coefficients are extracted from the carrier activations to form the next J-thought.
Termination: Recurrence stops when the J-thought stabilizes (low change in Frobenius norm) or a maximum cycle budget is reached. The final state conditions the autoregressive decoding of the answer.

2.3 Configurations

J-CoT-Zero: A training-free configuration using fixed interface components (mean pretrained embeddings, unit read strength). It requires no optimization specific to J-CoT.
J-CoT-Train: A trainable variant that optimizes the carrier embeddings and the read gate while keeping the underlying Transformer and J-lens dictionaries frozen.

3. Key Contributions

J-CoT Framework: Introduction of a recurrent reasoning framework that carries a vocabulary-indexed J-thought state across cycles, bridging the gap between explicit linguistic reasoning and dense latent recurrence.
Model-Native Read-Write Mechanism: Development of a mechanism that utilizes J-space coordinates to transport state without decoded rationales or full hidden-state recurrence, allowing the model to preserve information difficult to express in natural language while maintaining a structured interface.
Empirical Validation: Comprehensive evaluation across mathematical reasoning, scientific QA, code generation, and structured path reasoning, demonstrating that J-CoT matches or exceeds strong baselines.

4. Experimental Results

Experiments were conducted on the Qwen3-8B-Base backbone (and scaled to larger models) across eight benchmarks: GSM8K, MATH-500, AIME 2024, GPQA-Diamond, HumanEval+, MBPP+, LiveCodeBench, and CRUXEval.

Performance vs. Baselines:
- J-CoT-Train achieved the highest scores across all evaluated tasks, outperforming the strongest latent-reasoning baseline (SIM-Coconut) by an average of 2.7 percentage points (50.2% vs. 47.5%).
- J-CoT-Zero (training-free) matched or exceeded SIM-Coconut on every benchmark, improving the average score to 47.9% without any J-CoT-specific optimization.
- Both J-CoT variants significantly outperformed standard linguistic CoT and PS+ baselines.
Scaling: J-CoT exhibits positive scaling with both model capacity (from 7B to 405B parameters) and reasoning depth (adaptive cycle budgets). Larger models benefit more from additional reasoning cycles.
Interface Spectrum: An interpolation experiment between dense latent states ( $\lambda=0$ ), J-thoughts ( $\lambda=0.5$ ), and explicit linguistic states ( $\lambda=1$ ) revealed a non-monotonic trend. Performance peaked at the J-CoT configuration ( $\lambda=0.5$ ), suggesting it offers an optimal balance between preserving alternative reasoning paths (unlike explicit CoT) and reducing invalid-path persistence (unlike dense latent states).
Ablations: Removing J-thought reading or collapsing the J-thought to a single carrier significantly degraded performance, validating the necessity of the multi-carrier, vocabulary-indexed design.

5. Significance and Claims

The paper claims that J-CoT demonstrates the viability of model-native recurrent interfaces as a superior alternative to both fully verbalized and fully dense intermediate states.

Linguistic Grounding without Verbalization: The results support the hypothesis that a vocabulary-indexed coordinate system (J-space) allows for the extraction and reuse of internal computations without the overhead of generating fluent intermediate text.
Efficiency and Structure: By avoiding the commitment to a specific linguistic interpretation at every step, J-CoT allows the model to carry forward uncertain or distributed computations more effectively than explicit CoT, while the vocabulary indexing provides a selection mechanism absent in dense latent methods.
Training-Free Potential: The strong performance of J-CoT-Zero suggests that the pretrained Transformer already contains a usable J-space coordinate system, requiring no additional training to leverage this recurrent capability.

The authors conclude that J-CoT motivates further study into such interfaces, offering a path to more efficient and structured reasoning in large language models.