💻 computer science

A Node-by-Node White-Box Walkthrough of a Quantized Hybrid Gated DeltaNet Language Model: Instrument Calibration and the Information Trajectory from Embedding to Output

This paper presents a node-by-node white-box analysis of a quantized hybrid Gated DeltaNet language model using a custom, fully instrumented inference engine, demonstrating its calibration against standard implementations and providing reproducible, step-by-step observations of information flow from input embedding to output token while identifying open questions at the intersection of neural network theory and other scientific fields.

Original authors: Wenlong Li

Published 2026-09-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wenlong Li

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

To understand how a computer learns to speak, scientists have long treated the machine as a black box. They feed it a sentence and watch what comes out, or they peek inside at the final stage of thinking to guess what the machine was doing. But this approach leaves a gap. It is like trying to understand a clock by only looking at the time it displays, without ever seeing the gears turn. The new study by independent researcher Wenlong Li takes a different path. Instead of guessing from the outside, the researcher built a custom tool that lets them watch every single step of the machine's thinking process as it happens. This work focuses on a specific type of artificial intelligence that combines two different ways of processing information: one that remembers the past in a compact, efficient way, and another that looks back at everything it has seen so far. By watching a single word travel through the entire system, the study reveals exactly how the machine transforms a simple input into a complex answer, showing that the most important work happens not at the beginning, but at the very end.

The subject of this investigation is a language model, a computer program trained to predict the next word in a sentence. The researcher used a version of this model that has been compressed to fit on standard computer hardware, a process known as quantization, which reduces the precision of the numbers the machine uses to think. To ensure their custom tool was accurate, the researcher first tested it against a widely trusted, independent version of the same model. They ran the same sequence of words through both systems and compared the results. The two systems agreed on the vast majority of their choices, with their internal calculations matching closely enough to confirm that the custom tool was a reliable measuring instrument. This calibration was crucial, as it meant the researcher could trust the detailed observations that followed.

The journey begins when a single word, represented as a number, enters the machine. At this very first moment, the machine simply holds the meaning of that word. As the data moves into the first layer of the network, it is normalized, a process that adjusts its size to keep the calculations stable. At this stage, the machine is looking only at the current word; it has not yet remembered anything from the past. The first major transformation happens in the layers dedicated to the machine's memory. Here, the machine uses a short window to look at the few words that came immediately before, and it updates a large, internal memory matrix. This matrix acts like a dynamic notebook where the machine writes down new information and decides what to keep or discard based on how important it seems. The study found that this memory system does not change wildly with every new word; instead, it operates with a steady, controlled rhythm, keeping some memories for a long time while letting others fade away quickly.

As the information travels deeper into the network, it passes through layers that pay attention to the entire history of the conversation, not just the recent past. These layers connect the current word to everything that came before it, allowing the machine to understand context and relationships across the whole sentence. The researcher observed that the machine does not simply add new information to the old; it continuously reshapes the entire representation of the sentence. The original shape of the input word disappears quickly, replaced by a more abstract form that carries the meaning without the surface details. This abstract state persists for most of the journey through the network, a long middle section where the machine is organizing and refining its understanding without yet committing to a specific answer.

The most striking discovery of the study is how the machine finally decides what to say. For the majority of its depth, the machine moves the information away from the original input, keeping it in a state of abstract potential. It is only in the final few layers that the machine rapidly aligns this abstract state with the specific output it needs to produce. The study measured how much the data changed at each step and found that the most dramatic shift occurs at the very end, just before the machine selects the next word. This final calibration step adjusts the scale of the information so it can be translated back into a list of possible words. The machine spends most of its effort moving away from the input and only concentrates its decisive work on forming the output in the last moments.

The researcher also noted that while the machine's behavior is clear and measurable, some of the underlying reasons for its stability and efficiency are not yet fully explained by existing theories in physics or mathematics. The machine operates in a state that is balanced on the edge of stability, constantly updating and decaying information in a way that current scientific frameworks struggle to describe perfectly. The study does not claim to have solved these deep theoretical questions. Instead, it provides a precise, step-by-step record of what the machine actually does, releasing all the data and tools used so that others can verify the findings. By opening the black box and watching the gears turn, the researcher has shown that the path from a simple input to a complex answer is a long, abstract journey that only resolves into a clear decision at the very end.

Problem Statement

Current mechanistic interpretability of Large Language Models (LLMs) typically operates from the "outside." Researchers attach probes to hidden states, utilize the "logit lens" to project activations, or perform activation patching. These methods are constrained by the limited set of tensors exposed by standard inference engines and the opacity of compiled kernel launches. Consequently, when results diverge—such as after quantization—it is often ambiguous whether the model's behavior has genuinely changed or if the probing methodology is tracking different numerical paths. There is a lack of a "white-box" instrument that allows for the inspection of every mathematical operation (operator node) within a quantized, hybrid architecture during a single forward pass.

Methodology

The author developed a custom, graph-executing inference engine designed to treat the model as an explicit directed graph of operator nodes.

Instrumentation: The engine loads a standard GGUF model file, converts weights into explicit nodes, and executes the graph on a GPU using CUDA kernels. Every node's input and output can be read from or written to the host on demand.
Subject Model: The study focuses on a 24-layer, Q4_K-quantized hybrid model (Qwen3.5-0.8B). The architecture interleaves 18 Gated DeltaNet (state-space/linear attention) blocks with 6 global-attention blocks. The model dimension is 1024, and input/output embeddings are tied.
Calibration: Before analysis, the engine was rigorously calibrated against llama.cpp (a widely used independent implementation) using the same GGUF file and token sequences.
- A 27B-parameter variant of the same family was used for primary calibration over a 53-token sequence, achieving a mean cosine similarity of 0.9878 and 92.45% top-1 agreement.
- Disagreements were decoded and traced to specific causes (e.g., chat template end-token elevation in the reference implementation), confirming the engine's fidelity.
- The 0.8B walkthrough model was also calibrated directly against llama.cpp over 49 positions, achieving a mean cosine similarity of 0.945 and top-1 agreement of 0.878; errors were localized to late answer-option positions rather than the early layers examined.
Measurement Protocol: The study tracks a single token through the network under two conditions: "solo" (single token, no history) and "context" (five preceding tokens). This isolates where historical information enters the computation. Metrics include cosine similarity (direction), per-element RMS (scale), and logit lens projections.

Key Contributions

The paper provides a complete, operator-by-operator record of a token's forward path in a quantized hybrid model. The contributions are factual and observational rather than theoretical:

A Node-Readable Engine: A reproducible tool that exposes every operator node, removing ambiguity about numerical paths.
Operational Record: A step-by-step account of computation from embedding to output, including short convolutions, gated memory updates, and global attention.
Quantitative Measurements: Data on the short convolution window, forget/write gates, the associative memory matrix, and the evolution of the residual stream.
Identification of Theoretical Gaps: An explicit list of points where existing frameworks (physics, information theory, mathematics) do not yet directly cover the observed phenomena.

Key Results and Observations

The walkthrough reveals an asymmetric trajectory of computation:

Entry and Normalization: At the first block, entry normalization acts solely on the current token (cosine 1.0 between solo and context). History does not enter until subsequent layers.
State-Space Sub-layers (Gated DeltaNet):
- History Entry: The recurrent scan (memory matrix) is the sole gateway for prior tokens; all projection nodes remain identical to the solo run until the scan occurs.
- Throttle: The sub-layer operates with a "small throttle," adding small contributions (RMS ~0.03–0.08) to the residual stream at each step, relying on accumulation over depth.
- Gates: Forget gates form a fixed spectrum of time scales (ranging from ~0.001 to 1.0) configured in the weights, rather than varying dynamically with the token.
- Memory Dynamics: The associative memory matrix ( $S$ ) stays bounded. It grows from a zero state toward a plateau (norm ~1.06), where the rate of writing and decay approaches a balance.
Global Attention:
- Like the state-space layers, attention operates at a small throttle.
- Context reaches this layer via three routes: upstream state-space history, Rotary Position Embeddings (RoPE), and the Key-Value cache.
Residual Stream Evolution:
- Early Detachment: The cosine similarity between the residual stream and the input embedding drops sharply (from 1.0 to ~0.27) within the first two to three blocks. The original surface form is removed quickly.
- Abstract Middle: Blocks 3–13 operate in a region where the stream is neither aligned with the input nor the output; the logit lens reads no high-confidence tokens.
- Late Alignment: Significant reorganization and alignment with the final output occur only in the last few blocks (14–23). The final normalization step accounts for the largest magnitude change (norm ~89.6) to calibrate the stream for readout.
Final Output: The decision is completed at the very end. The network spends most of its depth moving the representation away from the input and through an abstract region, concentrating the work of organizing the output in the final layers.

Significance and Claims

The paper explicitly states it does not advance a unified theory of intelligence, stability, or computation. Its significance lies in providing a calibrated, reproducible instrument and a factual record of how known components execute in a quantized hybrid build.

The author highlights that at several points, the observed phenomena sit at the boundaries of existing disciplines:

Physics: The system operates near a stability boundary but does not collapse cleanly under standard equilibrium finite-size scaling, suggesting a need for non-equilibrium frameworks.
Information Theory: The "fit-then-compress" model is difficult to assert due to finite-sample floors in estimators; the intermediate region's information dynamics remain unclear.
Mathematics: The recurrent states involve non-normal operators where spectral radius and operator norms differ, and rigorous error bounds for quantized recurrence are not yet derived.
Engineering: Issues such as numerical drift in long sequences and draft-acceptance decay in long contexts are recorded as open problems rather than resolved.

The paper concludes that the data and scripts are released to allow other fields to pursue these open questions, positioning the work as a foundational empirical report rather than a theoretical resolution.

Problem Statement

Methodology

Key Contributions

Key Results and Observations

Significance and Claims

More like this