🧬 biology

Probing and steering biology across Boltz-1s trunk-diffusion boundary

This paper analyzes how biological information flows from the trunk to the diffusion module in the Boltz-1 protein structure predictor, revealing that while secondary structure and sequence chemistry are linearly decodable in the trunk, only secondary structure transfers effectively to the diffusion module, and crucially, that linear decodability of features like helices does not guarantee they can be causally steered to alter the model's output.

Original authors: Piotr Jedryszek, Tongmeng Xie, Adam Winnifrith, Alexander Hasson, Weronika Ślesak, George Wicks, Toby Winnifrith, Oliver M. Crook

Published 2026-08-13

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Piotr Jedryszek, Tongmeng Xie, Adam Winnifrith, Alexander Hasson, Weronika Ślesak, George Wicks, Toby Winnifrith, Oliver M. Crook

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

The Blueprint and the Builder: How AI "Thinks" About Proteins

Imagine trying to build a complex Lego castle. You have a master architect who looks at your list of bricks (the instructions) and figures out the overall shape, the color scheme, and where the towers should go. Then, you have a builder who takes those plans and actually snaps the bricks together, piece by piece, until the castle stands. For a long time, scientists have been fascinated by how these two roles work in nature. In the real world, proteins are the tiny machines that keep us alive, and their shape determines what they do. For years, we didn't know how to predict their shapes just from their instructions. But recently, powerful AI models have learned to do this, acting like both the architect and the builder in one.

These AI models work in two distinct stages. First, there's the "Trunk," a deep thinking engine that processes the sequence of amino acids (the instructions) and figures out the general geometry and chemistry of the protein. It's like the architect sketching the blueprint. Second, there's the "Diffusion Module," a generative engine that takes that blueprint and slowly, step-by-step, builds the final 3D structure, atom by atom. This is the builder snapping the bricks. The big question scientists have been asking is: as the information passes from the thinking architect to the building machine, does it change? Does the builder still "know" the chemical details, or does it only care about the shape? Understanding this helps us know if these AI models are truly understanding biology or just memorizing patterns, and whether we can actually "steer" them to design new things.

The Great Split: Where the Blueprint Meets the Builder

In this study, researchers took a close look inside a specific AI model called Boltz-1 to see what happens as information crosses the boundary between the "Trunk" (the thinker) and the "Diffusion Module" (the builder). They treated the AI like a black box they could peek inside, using special tools to read the model's "thoughts" (activations) at every layer.

The Architect Knows Everything
When they looked at the Trunk, they found it was incredibly well-informed. It held onto two main types of information:

Geometry: The physical shape, like whether a part of the protein is a spiral (helix), a flat sheet (strand), or a floppy mess (coil/disorder).
Sequence Chemistry: The specific chemical identity of the building blocks, such as which amino acid is where, or if there are special chemical tags like signal peptides or disulfide bonds.

The researchers could easily "read" both of these from the Trunk's internal state. It was like the architect had a perfect, detailed list of every brick's color and type, along with the final shape.

The Builder Loses the Details
However, as the information crossed the boundary into the Diffusion Module, a fascinating split occurred. The builder kept the geometry perfectly intact. Even as the model started constructing the final 3D shape, it still clearly "knew" where the spirals and sheets were. But the chemistry started to fade away. The specific chemical identities, like signal peptides and disulfide bonds, became much harder to detect. By the time the builder finished its work, the chemical details were largely gone, while the shape remained sharp.

It's as if the builder received the architect's plan, understood exactly where the walls and towers should go, but forgot what color the bricks were supposed to be. The model uses the chemical details to figure out the shape, but once the shape is being built, it doesn't seem to need to keep those chemical details in its active memory.

Can We "Steer" the AI?

The researchers then asked a bold question: If we can read these thoughts, can we also change them? They tried to "steer" the model by nudging the Trunk's final thought before it handed the blueprint to the builder. They added a tiny push in the direction of "make more spirals" or "make more sheets."

The Success: When they nudged the model to make more spirals (helices) or floppy parts (coils), the model listened! The predicted structure changed exactly as they hoped, with more spirals or coils appearing. This proved that the model was causally using those specific directions to build the shape.
The Surprise: When they tried to do the same thing for sheets (strands), even though the model could easily read the "sheet" direction (it was highly predictable), the nudge did nothing. The amount of sheets in the final structure didn't change.

This suggests a crucial limit: just because a model can predict something doesn't mean you can control it by changing that specific part of the brain. The researchers suspect that "sheets" depend on how pairs of amino acids interact with each other (a detail handled in a different part of the model's memory that they didn't touch), whereas spirals are more local and can be controlled directly.

The Trap of Missing Labels

The paper also uncovered a tricky problem in how we test these AI models. Scientists often check if the AI is right by comparing its predictions to existing databases of protein shapes. However, these databases are often "sparse"—meaning they only have labels for the parts of the protein that scientists have already studied in a lab.

The researchers found that when they tested the AI against these sparse labels, it looked like the AI was making a lot of mistakes (false positives). But when they compared the AI to a "dense" label set (where every single part of the protein is labeled), the AI looked much smarter. The "mistakes" were actually just correct predictions for parts of the protein that the database simply hadn't labeled yet. It's like a student getting an A on a test, but the teacher marks them wrong because the answer key only lists questions the teacher decided to ask, ignoring the rest of the student's correct work. This means we might be underestimating how good these models really are.

What This Means for the Future

The study concludes that while AI models like Boltz-1 are powerful, they aren't magic. They have a clear division of labor: the "thinking" part holds all the chemical and structural details, but the "building" part focuses almost entirely on geometry. Furthermore, being able to read a feature doesn't guarantee you can control it.

The researchers suggest that if we want to use these models to design new proteins, we need to be careful. We can't just assume that because a model "knows" a chemical feature, we can force it to use that feature to build something new. And when we test these models, we need to be careful not to judge them harshly just because the reference data is incomplete. It's a reminder that even the smartest AI has its own unique way of thinking, and we need to learn its language before we can truly steer it.

Technical Summary: Probing and Steering Biology Across Boltz-1's Trunk–Diffusion Boundary

Problem Statement
Modern protein structure predictors, such as AlphaFold3 and Boltz-1, utilize a hybrid architecture comprising a representational "trunk" (e.g., the Pairformer) that processes sequence and context, and a "diffusion module" that generates atomic coordinates. While these models achieve high accuracy, the nature of biological information as it transitions across this architectural boundary remains poorly understood. Specifically, it is unclear how the representation of sequence chemistry (e.g., amino acid identity, signal peptides) versus geometric features (e.g., secondary structure) evolves from the trunk to the diffusion module, and whether linearly decodable signals in these representations correspond to causal levers for steering the model's output.

Methodology
The authors analyzed Boltz-1 using a combination of linear probing, sparse autoencoders (SAEs), and causal interventions.

Data Extraction: Per-residue activations were extracted from the Pairformer trunk (48 layers) and the diffusion module (22 layers, 200 sampling steps). Comparisons were normalized by fractional depth.
Probing & SAEs: The team trained logistic probes and TopK SAEs (with 256 or 2048 latents) on demeaned activations. They evaluated four readout types: raw probes, probes on SAE latents, single best SAE features, and single best raw neurons.
Labels: Evaluation utilized three label sources: sparse SwissProt annotations, dense DSSP labels derived from AlphaFold DB structures, and dense DSSP labels from Boltz-1's own predictions (as a self-consistency control). Concepts were categorized into "geometry" (helix, strand, coil, disorder) and "sequence chemistry" (amino-acid identity, signal peptides, disulfide bonds).
Causal Steering: To test causality, the authors intervened on the final trunk single representation ( $s$ ), which conditions the diffusion module. They added specific direction vectors (derived from probes or SAEs) to $s$ and measured the resulting change in predicted secondary structure content against matched-norm random controls.

Key Results

Representational Split: The Pairformer trunk retains linear access to both geometric features and sequence chemistry. In contrast, the diffusion module preserves geometric information (secondary structure remains decodable with high F1 scores) but strongly attenuates sequence chemistry. For instance, signal peptide decodability drops from an F1 of 0.76 in the trunk to 0.35 in the diffusion module, and amino-acid identity drops from 1.00 to 0.66.
Decodability vs. Causal Influence: Linear decodability does not guarantee causal steerability. While adding "helix" and "coil" directions to the trunk representation dose-dependently increased the respective secondary structure content in the final prediction, adding a highly predictive "strand" direction (F1 = 0.82) produced no measurable increase in strand content. Instead, it converted helix to coil. The authors hypothesize that $\beta$ -strand formation relies on pairwise residue information ( $z$ ) rather than single-residue information ( $s$ ), and since information flows $z \to s$ but not vice versa in Boltz-1, steering $s$ cannot influence strand formation.
Annotation Bias: Evaluating probes against sparse SwissProt annotations significantly underestimates model performance compared to dense DSSP labels. The authors demonstrate that "missing" labels in sparse datasets cause correct predictions to be scored as false positives, artificially lowering precision. For example, helix probe scores drop from 0.85 (DSSP) to 0.42 (SwissProt).
Supervised Probes vs. SAEs: Where ground-truth labels exist, supervised probes consistently outperform single SAE features in both decoding accuracy and steering reliability. SAEs showed no direct advantage over supervised probes for known concepts but may remain valuable for hypothesis generation regarding unlabeled features.

Significance and Claims
The paper claims to provide the first systematic contrast of the trunk/diffusion boundary in structure prediction models, revealing that the two modules expose different information spaces. The authors argue that:

Linear decodability is not sufficient evidence of causal use: A signal can be perfectly decodable yet causally inert at the intervention site tested.
Evaluation metrics are sensitive to label sparsity: Standard evaluation against sparse biological annotations (like SwissProt) inflates false-positive rates and underestimates the true alignment between model representations and biological concepts.
Methodological Recommendations: Claims about represented information must specify the readout class and avoid equating decodability with causal influence. Furthermore, causal conclusions should be restricted to the specific intervention sites and families tested.

The authors conclude that while SAEs are useful for generating hypotheses about unlabeled features, supervised probes remain the more reliable tool for interpreting and steering biological ML models where labels exist. They release the trained SAEs, activations, and analysis code to facilitate further investigation into these representational boundaries.