💬 NLP

Universal Activation Verbalizer: A Unified Framework for Cross-Model Activation Explanation

The paper introduces the Universal Activation Verbalizer (UAV), a unified framework that employs a shared decoder and lightweight adapters to enable cross-model activation explanation across heterogeneous donor models, achieving performance competitive with self-explanation baselines while supporting efficient adapter-only transfer.

Original authors: Haiyan Zhao, Zirui He, Guanchu Wang, Ali Payani, Yingcong Li, Mengnan Du

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Haiyan Zhao, Zirui He, Guanchu Wang, Ali Payani, Yingcong Li, Mengnan Du

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Black Box" Translator

Imagine a large language model (like a super-smart robot brain) as a massive factory. Inside this factory, information flows through conveyor belts and machines. At certain points, the machines produce "activations"—these are just raw, electrical signals or numbers that represent what the robot is thinking at that exact moment.

The problem is that these numbers are gibberish to humans. We can't read them.

Previously, scientists tried to figure out what these numbers meant by teaching the robot to explain its own thoughts. It's like teaching a specific person to translate their own secret code. But if you wanted to understand a different robot's code, you'd have to start all over and teach that new robot to translate itself. It was slow, expensive, and didn't work across different types of robots.

The Solution: The "Universal Translator" (UAV)

The authors created a new tool called UAV (Universal Activation Verbalizer). Think of UAV as a universal translator that can listen to any robot factory and explain what its internal signals mean in plain English.

Here is how it works, broken down into three simple parts:

1. The Adapter: The "Universal Ear"

Every robot factory speaks a slightly different "language" of numbers. To understand a new robot, UAV uses a small, flexible piece of software called an Adapter.

Analogy: Imagine the Adapter is a pair of universal earbuds. When you plug them into a new robot, they instantly convert that robot's weird, specific electrical signals into a standard "soft language" that a translator can understand.
The paper tested two types of these earbuds: a simple "MLP" (a straight-line converter) and a "Q-Former" (a more complex, attention-based converter). They found the Q-Former worked best because it could hold onto more details from the robot's signals.

2. The Decoder: The "Storyteller"

Once the Adapter converts the signals into the standard language, they are fed into a Decoder (a large language model).

Analogy: The Decoder is the storyteller. It takes the standard language from the earbuds and turns it into a natural sentence like, "This signal means the robot is thinking about a cat," or "This signal represents a fact about the year 1995."
The paper found that making the storyteller bigger (using a larger model) generally made the explanations better, but only up to a point.

3. The Two-Step Training Process

To teach this system, the authors used a two-step training method:

Step 1: The Warm-up (Reconstruction). First, they taught the Adapter to simply look at a signal and rebuild the original text that created it. It's like teaching the earbuds to say, "If I hear this beep, the original text was 'Hello'." This ensures the Adapter knows how to translate the signals accurately.
Step 2: The Explanation (Instruction). Next, they taught the whole system to answer questions. Instead of just rebuilding text, they asked, "What is the main idea of this text?" or "What is the profession of this person?" This taught the system to be a helpful explainer, not just a mirror.

The Big Breakthrough: "Plug-and-Play"

The most exciting part of the paper is a feature called Adapter-Only Transfer.

The Old Way: To understand Robot A, you train a translator. To understand Robot B, you have to retrain the entire translator from scratch.
The UAV Way: You train the "Storyteller" (the Decoder) once using Robot A. Then, if you want to understand Robot B, you freeze the Storyteller (keep it exactly as is) and only train a new set of "Earbuds" (the Adapter) for Robot B.
Analogy: Imagine you have a master translator who speaks perfect English. You don't need to retrain the translator to understand a new language; you just give them a new pair of earbuds tuned to that new language. The translator stays the same, but the earbuds do the heavy lifting of conversion.

What Did They Find?

The researchers tested this on different types of robots (Llama, Gemma, Yi, Qwen) and different tasks (answering trivia, summarizing text, classifying emotions).

It Works Everywhere: The system could explain the thoughts of completely different robot families, not just the one it was originally trained on.
It's Competitive: Even though it was translating other robots, it was just as good at explaining things as robots that were trained to explain themselves.
The Roles are Clear:
- The Adapter (Earbuds) is responsible for grabbing the specific facts and details from the robot's brain.
- The Decoder (Storyteller) is responsible for knowing how to answer questions and follow instructions.
Size Matters (But not everything): Bigger storytellers generally did a better job, but simply making them huge didn't always guarantee perfection. The quality of the "Earbuds" (the Adapter) was just as important.

Summary

In short, the authors built a universal translator that can listen to the internal thoughts of almost any AI model and explain them in plain English. They proved that you don't need to retrain the whole system for every new AI; you just need to swap out the "earbuds" (the Adapter) while keeping the "storyteller" (the Decoder) the same. This makes understanding AI much faster, cheaper, and more flexible.

Technical Summary: Universal Activation Verbalizer (UAV)

Problem Statement

While Large Language Models (LLMs) have achieved significant success through representation learning, understanding the specific information encoded in their internal activations remains a central challenge. Existing activation verbalization methods—where a model describes the content of a hidden state in natural language—are predominantly limited to self-explanation. In these approaches, a model is trained to explain only its own activations. This restriction prevents the reuse of verbalizers across different model families, architectures, or scales, making the analysis of large "donor" models inefficient and non-generalizable. Furthermore, training-free methods (e.g., patching activations directly into prompts) often fail to recover fine-grained factual and semantic information.

Methodology: The UAV Framework

The authors propose the Universal Activation Verbalizer (UAV), a unified framework designed to explain activations from heterogeneous donor models using a shared decoder. The framework operates on the premise that a lightweight adapter can map donor activations into the embedding space of a decoder model, enabling cross-model verbalization.

Architecture

UAV consists of two primary components:

Trainable Adapter ( $\mathcal{A}$ ): This module bridges the donor activation space and the decoder's embedding space. It maps a specific donor activation $h_i^\ell$ $h_{i}^{ℓ}$ (from layer $\ell$ $ℓ$ at position $i$ $i$ ) into a sequence of $n$ $n$ continuous "soft tokens" ( $z_{1:n}$ $z_{1 : n}$ ). The paper investigates two adapter architectures:
- MLP-style: A projection network with optional bottleneck layers.
- Q-Former-style: An attention-based design using learnable query tokens to extract information from context slots derived from the donor activation.
Decoder Model ( $\mathcal{M}_v$ ): A frozen or partially fine-tuned language model (e.g., Qwen-family) that generates natural language explanations. The soft tokens generated by the adapter are prepended to the textual prompt as an activation-conditioned prefix.

Training Strategy

UAV employs a two-stage training process:

Activation-to-Text Alignment (Stage 1): The adapter is trained to reconstruct the input text prefix $\langle t_0, \dots, t_i \rangle$ from the donor activation $h_i^\ell$ using teacher forcing. The decoder is frozen during this stage. This serves as a warm-up to align the activation space with the decoder's embedding space.
Instruction-Following Alignment (Stage 2): The model is fine-tuned to answer questions about the input text (e.g., "What is the gist?" or "What was the profession?"). This stage involves joint training of the adapter and decoder-side parameters (specifically LoRA) to improve the model's ability to follow explanation-oriented instructions.

Adapter-Only Transfer

A key innovation is the Adapter-Only Transfer strategy. After performing full two-stage adaptation on a source donor, the decoder-side LoRA is frozen. For a new donor model, only a new adapter is trained from scratch while keeping the decoder backbone and the learned LoRA fixed. This allows the reuse of the decoder's explanation-oriented behavior across different donor models without retraining the entire system.

Key Contributions

Unified Framework: The proposal of UAV, an adapter-decoder framework that enables cross-model activation verbalization across heterogeneous model families (e.g., Llama, Gemma, Yi, Qwen) and scales.
Adapter-Only Transfer: The introduction of a transfer mechanism that freezes the decoder-side LoRA, allowing for efficient adaptation to new donors by training only the adapter.
Systematic Analysis: Comprehensive experiments and ablations demonstrating that UAV supports activation explanation across model families while remaining competitive with strong self-explanation baselines.

Experimental Results

The authors evaluated UAV on generative activation-grounded QA tasks covering classification, fact retrieval, and gist summarization.

Performance vs. Baselines: UAV outperforms training-free methods (SelfIE, PatchScope) significantly, particularly in fact retrieval where patching methods struggle with fine-grained information. Compared to training-based self-explanation baselines (LatentQA, Activation Oracle), UAV remains competitive, often achieving the best overall ROUGE-L and BERTScore in self-explanation settings.
Cross-Model Capability: A shared decoder (e.g., Qwen3-4B) successfully verbalizes activations from diverse donors (Llama-3.1-8B, Gemma-3, Yi-1.5-34B). While self-decoding (using the donor as its own decoder) generally yields slightly better results, the performance gap in cross-model verbalization is moderate, demonstrating the framework's universality.
Decoder Size: Larger decoders (up to 14B) generally improve performance, suggesting better conversion of aligned activation signals into natural language, though the trend is not strictly monotonic.
Component Analysis (Ablations):
- Decoder-side Tuning: Primarily improves task-level behavior and instruction following (e.g., classification patterns).
- Adapter: Essential for recovering input-specific factual and semantic information. Without the trained adapter, the model cannot reliably extract fine-grained details from activations.
- Stage-1 Warmup: The activation-to-text alignment stage is a critical prerequisite; removing it consistently degrades performance.

Significance and Claims

The paper claims that UAV addresses the inefficiency and lack of generalizability in current activation explanation methods. By decoupling the activation mapping (adapter) from the generation capability (decoder), UAV enables:

Reusability: A single explanation-oriented decoder can be adapted to multiple donor models.
Efficiency: The adapter-only transfer strategy reduces computational costs when analyzing new models.
Faithfulness: The framework provides a mechanism to recover factual and semantic information from activations that is often lost in training-free approaches.

The authors acknowledge limitations, noting that UAV still faces challenges in recovering fine-grained attributes from single activations and is currently limited to short-context inputs. Future work is proposed to improve fact retrieval and gist understanding through stronger alignment and richer supervision.