🤖 AI

Scaling-Aware Adapter for Structure-Grounded LLM Reasoning

The paper introduces Cuttlefish, a unified multimodal LLM that enhances structure-grounded reasoning and mitigates hallucinations by employing a Scaling-Aware Patching mechanism to adaptively scale query tokens based on structural complexity and a Geometry Grounding Adapter to inject explicit geometric cues into the model.

Original authors: Zihao Jing, Qiuhao Zeng, Ruiyi Fang, Yan Yi Li, Yan Sun, Boyu Wang, Pingzhao Hu

Published 2026-05-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zihao Jing, Qiuhao Zeng, Ruiyi Fang, Yan Yi Li, Yan Sun, Boyu Wang, Pingzhao Hu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Blind" Scientist

Imagine you have a brilliant scientist (a Large Language Model, or LLM) who has read every book ever written about chemistry and biology. They know the names of molecules, how proteins are spelled out in sequences, and the rules of life.

However, there's a problem: This scientist is blind to 3D shapes.

If you show them a flat list of ingredients (a chemical sequence), they can guess what the dish might taste like. But if you ask them, "Will this molecule fit into this specific lock?" or "Why does this protein fold into a spiral?", they might start guessing wildly. They might invent a shape that doesn't exist just to make the story sound good. In the paper, this is called "structural hallucination."

Existing AI models try to fix this by showing the scientist a picture of the molecule, but they usually squish that picture into a tiny, fixed-size summary (like trying to describe a massive cathedral using only 10 words). If the molecule is small, they waste words. If it's huge, they lose important details.

Cuttlefish is a new system designed to give this scientist "3D vision" without overwhelming them.

The Two Main Problems Cuttlefish Solves

1. The "One-Size-Fits-All" Trap (Scaling)

The Problem: Imagine you are a tour guide for a museum.

If you show a visitor a tiny key, you might spend 1 minute describing it.
If you show them a massive castle, you might spend 10 minutes.
Old AI models are like a strict tour guide who says, "I only have 1 minute to describe everything, no matter how big it is."
- For the key, they waste time.
- For the castle, they have to skip the towers, the dungeons, and the moat, leaving the visitor confused.

The Cuttlefish Solution: "Scaling-Aware Patching"
Cuttlefish uses a smart "gating" system. It looks at the instruction (the question) and the size of the molecule.

If the molecule is small, it picks a few key spots to describe.
If the molecule is huge (like a giant protein), it dynamically expands its attention. It says, "Okay, this is complex; I need to zoom in on 50 different spots to get the details right."
Analogy: Instead of a rigid 1-minute speech, Cuttlefish is a flexible tour guide who adjusts the length of the tour based on how big the building is, ensuring no important detail is missed.

2. The "Imaginary Shapes" Problem (Hallucination)

The Problem: Without seeing the actual 3D shape, the scientist might say, "This protein looks like a sphere," when it's actually a flat sheet. They are guessing based on the text description alone.

The Cuttlefish Solution: "Geometry Grounding Adapter"
This part acts like a translator that converts the raw 3D coordinates (the actual X, Y, Z positions of every atom) into a language the scientist understands.

It doesn't just say "here is a molecule." It points to specific geometric features: "Look here, there is a sharp bend," or "Notice this cluster of atoms is far away from that one."
Analogy: It's like giving the blind scientist a pair of 3D glasses. Instead of guessing the shape, they can now "see" the geometry and say, "Ah, I see the spiral now. That explains why it works." This stops them from making up fake shapes.

How It Works (The "Cuttlefish" Metaphor)

The model is named Cuttlefish because, like the real animal, it is versatile and can change its shape to fit its environment.

The Body: It connects to a standard Large Language Model (the "brain").
The Arms: It has a special connector that can stretch or shrink.
- It grabs the "anchor points" of a molecule that are most important for the specific question asked.
- It grows "patches" around those anchors to cover the necessary area.
- It feeds this information into the brain, ensuring the brain sees the actual shape, not just a compressed summary.

What the Paper Claims (The Results)

The authors tested Cuttlefish on three types of biological entities: Molecules (small chemicals), Proteins (large biological machines), and Nucleic Acids (DNA/RNA).

Better Accuracy: Cuttlefish answered questions about these structures much better than models that only read text or models that use the old "fixed-size" summaries.
Fewer Lies: It made significantly fewer "hallucinations" (inventing fake shapes or properties). When asked about how a molecule absorbs into the body, it used the actual 3D shape to give the right answer, whereas others guessed.
Efficiency: It didn't waste computer power. It used just enough "tokens" (words describing the shape) for the job—few for small things, more for big things—rather than using a fixed amount for everything.
Unified: It works as a single system for all three types of biology (molecules, proteins, DNA), whereas previous models often needed separate systems for each.

Summary

Cuttlefish is a tool that teaches AI to "see" the 3D shapes of molecules and proteins. It solves the problem of AI getting overwhelmed by big shapes or ignoring small details by dynamically adjusting how much attention it pays. By forcing the AI to look at the actual geometry, it stops the AI from making up fake structures, leading to more reliable scientific reasoning.

Technical Summary: Scaling-Aware Adapter for Structure-Grounded LLM Reasoning (Cuttlefish)

1. Problem Statement

Large Language Models (LLMs) are increasingly applied to reason over 2D and 3D scientific structures (molecules, proteins, nucleic acids). However, existing methods face two critical limitations that hinder generalized all-atom reasoning:

Fixed-Length Bottlenecks (Budget Scaling): Current architectures typically rely on fixed-length query connectors (e.g., Q-Former style) or sequence-based tokenization. These approaches compress variable-size structural graphs into a constant token budget. As structural complexity increases (e.g., larger proteins or nucleic acids), this leads to over-compression of geometric features, degrading performance. Conversely, simply increasing the token budget for all inputs leads to inefficient allocation, diluting attention on small entities.
Structural Hallucinations: Models relying solely on sequence inputs (e.g., SMILES, residue sequences) lack explicit geometric grounding. Without verifiable 3D structural evidence, LLMs tend to generate geometric rationales that are not entailed by the conditioning context, leading to hallucinations regarding functional groups, mechanisms, or long-range interactions.

The paper argues that a unified interface for heterogeneous all-atom entities is missing, necessitating a rethinking of how structural evidence is selected and exposed to the language model.

2. Methodology: Cuttlefish

The authors introduce Cuttlefish, a unified multimodal LLM designed to ground language reasoning in geometric cues while scaling modality tokens with structural complexity. The architecture consists of three primary components:

A. Encoder (SE(3)-Equivariant EGNN)

Input atomic coordinates and features are first encoded using an Equivariant Graph Neural Network (EGNN). This ensures the representation is invariant to rotations and translations, producing atom-level modality embeddings. The encoder is pretrained via a masked reconstruction task (predicting element identity, pairwise distances, and direction noise) to learn spatial relations.

B. Scaling-Aware Patching

To address the fixed-budget bottleneck, Cuttlefish employs an instruction-conditioned gating mechanism:

Mass-Based Anchor Selection: Given an instruction embedding and node embeddings, a scoring gate computes per-node relevance logits. Anchors are selected adaptively based on a cumulative mass threshold ( $\rho$ ). This ensures the number of query tokens ( $k_g$ ) scales with the structural complexity of the entity, selecting only the most instruction-relevant atoms.
Soft Patch Growth: Selected anchors expand into structural patches via a soft assignment mechanism. This fuses spatial proximity (Euclidean distance) with semantic bias (instruction relevance).
In-Patch Weighted Pooling: Node features within each patch are aggregated using membership weights to generate structural query tokens. This allows the model to preserve fine-grained geometric signals for complex regions while compressing less relevant areas.

C. Geometry Grounding Adapter

To mitigate structural hallucinations, the adaptive tokens are refined and injected into the LLM:

Cross-Attention Retrieval: The patch tokens undergo cross-attention with the full modality embeddings (all atoms) through a series of fusion blocks. This allows the queries to retrieve high-resolution geometric cues abstracted during the patching phase.
Injection: The refined modality tokens are projected into the LLM's embedding space and inserted at designated placeholder positions within the instruction sequence. This exposes explicit geometric cues to the LLM, conditioning its output on verifiable structure.

D. Training Protocol

The model is trained in three stages:

Encoder Pretraining: The EGNN is pretrained on all-atom spatial graphs.
Modality Alignment Tuning: The EGNN and LLM are frozen; only the connector (Patching and Adapter) is optimized on the GEO-AT dataset (a new all-atom instruction dataset introduced by the authors).
LLM Adaptation Tuning: The LLM is unfrozen with a smaller learning rate to accommodate the new modality injection distribution while preserving general language capabilities.

3. Key Contributions

Scaling-Aware Patching: An instruction-conditioned anchor patch growing mechanism that ensures the modality token count scales with structural complexity, effectively mitigating fixed-length connector bottlenecks.
Geometry Grounding Adapter: A module that injects verifiable geometric cues into the LLM via modality tokens, explicitly conditioning the model on geometry to reduce structural hallucinations.
Cuttlefish Framework: A unified structure-aware LLM capable of reasoning over heterogeneous all-atom modalities (molecules, proteins, DNA, RNA) with strong performance across benchmarks.
GEO-AT Dataset: The introduction of the first all-atom instruction dataset encompassing molecules, proteins, and nucleic acids with atom-level coordinates and language annotations.

4. Experimental Results

Experiments were conducted on heterogeneous all-atom benchmarks including Mol-Instructions, DNA-Chat, and RNA-QA.

Performance vs. General LLMs: Cuttlefish significantly outperforms general LLM baselines (e.g., Llama-3, Qwen, Mistral) and reasoning-enhanced variants across all modalities. For instance, on the GEO-AT benchmark, Cuttlefish + Llama-3.1-8B achieved an average METEOR score of 0.428 and BERTScore of 0.864, compared to 0.186 and 0.694 for the backbone alone.
Performance vs. Modality-Specific Baselines: Cuttlefish achieves top-1 performance on most tasks, surpassing specialized models like Mol-Llama (molecules), ProtChatGPT (proteins), and ChatNT (nucleic acids).
Scaling Stability: Unlike fixed-length connectors that degrade as entity size increases, Cuttlefish maintains stable generation quality across varying graph node counts (from small molecules to large proteins), demonstrating favorable scaling characteristics.
Token Efficiency: Cuttlefish maintains a substantially lower structural token budget compared to linear-scaling baselines (e.g., ChatNT) while preserving performance, confirming the efficiency of adaptive token allocation.
Hallucination Reduction: In hallucination evaluations (functional group prediction, mechanism explanation, long-range interaction), Cuttlefish achieves the lowest hallucination rates (e.g., 0.04 for protein functional groups vs. 0.10 for ProtChatGPT) and highest answer rates.
Robustness: The model demonstrates graceful degradation under coordinate noise and remains effective even when experimental structures are replaced by AlphaFold2 predictions.

5. Significance and Claims

The paper claims that Cuttlefish establishes a unified framework for structure-grounded reasoning across all-atom modalities. Its significance lies in addressing the dual challenges of scalability and faithfulness:

Scalability: By replacing rigid fixed-length connectors with Scaling-Aware Patching, the architecture bypasses information bottlenecks, allowing the model to handle structural complexity without over-compression or context inefficiency.
Faithfulness: By using the Geometry Grounding Adapter to inject explicit geometric evidence, the model reduces the irreducible uncertainty (Bayes risk) associated with sequence-only inputs, thereby suppressing structural hallucinations.

The authors conclude that sequence-only inputs are insufficient for structure-conditioned understanding and that fixed-length connectors impose a fundamental bottleneck for capturing geometry-dependent semantics. They posit that joint modality training with adaptive token scaling is key to robust, unified multimodal scientific reasoning. The work is intended for general scientific modeling settings, with no broader societal impacts beyond those common to the research area.