💻 computer science

Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space

This paper proposes a unified geometric fingerprinting framework that analyzes the spectral energy and subspace alignment of LLM weight matrices to robustly trace model lineage and distinguish between independent, same-series, and shared-base models without requiring access to input data.

Original authors: Yiwei Chen, Bingqi Shang, Sijia Liu

Published 2026-08-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yiwei Chen, Bingqi Shang, Sijia Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where every time you build a robot, you leave behind a unique, invisible DNA strand inside its brain. In the fast-moving universe of Artificial Intelligence, these "robots" are called Large Language Models (LLMs)—super-smart computer programs that can write stories, solve math problems, and chat like humans. But here's the twist: these models aren't always built from scratch. Often, one company takes a giant, pre-made model, tweaks it, trains it on new data, and releases a "new" version. It's like taking a famous chef's secret recipe, adding a pinch of salt, and calling it your own. This creates a messy family tree where it's hard to tell who is related to whom, who copied whom, and where a model actually came from. This is the problem of "provenance": knowing the true origin of something. Scientists have tried to solve this by looking at how models act (like testing their answers), but that's tricky because the answers can change depending on how you ask the questions. So, the big question becomes: Can we look inside the model's brain (its weights) to find a permanent, unchangeable fingerprint that tells us its family history, without needing to ask it a single question?

This paper, titled "Who Built This Model?", says yes, we can. The authors, researchers from Michigan State University, propose a new way to trace the lineage of AI models by looking at the geometry of their internal weights. They treat the model's brain like a complex musical instrument and listen to two different "sounds" it makes: the loudness of the notes (spectral energy) and the direction the sound waves travel (subspace alignment).

First, they looked at the "loudness" or the overall energy of the model's weights. They found that this acts like a coarse-grained family tree. If two models are from completely different families (like a robot built by Company A vs. Company B), their "loudness" patterns are totally different. If they are from the same series (like a 3-billion-parameter version and a 7-billion-parameter version of the same model), their patterns are very similar. The researchers showed that by measuring this "spectral energy," they could easily tell if two models were strangers or distant cousins. This is a big deal because previous methods often got these two groups mixed up, failing to see the difference between a totally new model and a scaled-up version of an old one.

However, the "loudness" test hits a wall when the models are very close relatives. Imagine two models that started from the exact same base but then went through different training camps: one learned to be a math tutor, and the other learned to be a creative writer. Their "loudness" is almost identical, so the first test can't tell them apart. This is where the second part of the paper's magic happens: "subspace alignment." Instead of just listening to how loud the music is, the authors looked at the direction of the notes. They discovered that even when the overall volume is the same, the specific directions the model's brain waves point in change slightly depending on what the model learned. If a model was trained on a tiny dataset, its directional fingerprint is almost the same as the original. But if it was trained on a massive, diverse dataset, those directions shift noticeably.

The team tested this on over 110 different pairs of open-weight models. They found that their new method could do what old methods couldn't: it could separate the "strangers" from the "cousins," and then zoom in to tell the difference between "cousins" who had different life experiences (different training data or algorithms). For example, they showed that their method could detect that a model trained on 100% of a dataset had a different geometric fingerprint than one trained on only 10%, even though they started from the same base. They also found that different training algorithms (like DPO, PPO, and RAFT) leave distinct geometric scars on different parts of the model's brain.

In short, the paper suggests that AI models have intrinsic "biometrics" hidden in their weight space. By combining a check for global energy (to separate different families) and a check for directional geometry (to distinguish close relatives), we can build a reliable map of who built what, and how they evolved. This doesn't just satisfy curiosity; it offers a tool for checking the supply chain of AI, ensuring that we know the true origins of the models we use, and helping to govern their evolution as they become more complex. The authors emphasize that this works even when we don't have access to the training data, only the final model weights, making it a powerful tool for transparency in the AI world.

Technical Summary: Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space

Problem Statement

The rapid proliferation of open-weight Large Language Models (LLMs) developed through complex, multi-stage pipelines (pretraining, fine-tuning, alignment) has created intricate lineage relationships. Understanding these relationships is critical for model provenance, governance, and supply-chain integrity. However, existing methods for identifying model lineage face significant limitations:

Black-box approaches (e.g., prompt-response probing, watermarking) rely on input data, are sensitive to prompt design, and do not reveal intrinsic structural signatures.
White-box approaches (e.g., weight statistics, parameter distribution fingerprints) often reduce complex weight-space information to coarse summary statistics. As demonstrated in the paper, these methods frequently fail to distinguish between models with different origins or scales, particularly when distinguishing "independent-origin" models from "same-series" models.

The core research question is: What weight-space structures serve as reliable fingerprints for LLM lineage identification? The authors investigate this under a white-box setting where model parameters are accessible, but training data and metadata are not.

Methodology: Unified Geometric Fingerprinting Framework

The authors propose a unified geometric fingerprinting framework that analyzes weight matrices from two complementary perspectives to address hierarchical lineage discrimination. The framework defines three regimes of increasing shared information:

Independent-Origin (S1): Models trained independently without shared initialization, data, or pipelines.
Same-Series (S2): Models sharing architecture and training pipelines but differing in parameter scale.
Shared-Base (S3): Models derived from a common pretrained model that diverge through post-training (e.g., different datasets or algorithms).

The methodology decomposes weight matrices $W$ via Singular Value Decomposition (SVD) to extract two distinct signals:

1. Spectral Energy Analysis (Coarse-Grained Discrimination)

To distinguish between independent origins and same-series models, the authors analyze the spectral energy (global magnitude patterns) of weight matrices.

Mechanism: They compute the spectral trace fingerprint, defined as the sum of squared singular values ( $\sum \sigma_i^2$ ) for each layer and component type (e.g., Q, K, V, FFN).
Metric: A similarity score is calculated using Pearson correlation between the layer-wise trace fingerprints of two models, normalized via linear interpolation to handle depth differences.
Function: This captures global structural patterns, enabling the separation of model families and scales.

2. Subspace Alignment Analysis (Fine-Grained Discrimination)

To distinguish closely related models within the "Shared-Base" regime (e.g., variations in dataset scale or post-training algorithms), the authors analyze directional geometry.

Mechanism: They perform compact SVD on weight matrices and extract the top- $k$ left singular vectors ( $U_k$ ) to define dominant subspaces.
Metric: They compute the cross-subspace matrix $C = U_a^T U_b$ and analyze its singular values, which correspond to the cosines of principal angles between subspaces. The metric aggregates the smallest singular values (representing the worst-case alignment) to measure directional deviation.
Function: This captures geometric variations and subspace rotations induced by fine-tuning data scale and specific post-training algorithms, which spectral magnitude alone cannot detect.

Key Results

The authors conducted extensive experiments on over 110 diverse open-weight LLM pairs (including LLaMA, Qwen, and Gemma families).

Hierarchical Separation: The proposed Trace metric (spectral energy) successfully achieves coarse-grained separation. It clearly distinguishes independent-origin models from same-series models, a task where baselines like AWM (Accurate Weight Matrix) and PDF (Parameter Distribution Fingerprint) fail, often showing significant overlap or random-chance performance (AWM AUC = 0.256 vs. Trace AUC = 0.850 for S1 vs. S2).
Fine-Grained Sensitivity: The Subspace Alignment metric successfully discriminates within the shared-base regime.
- Data Scale: It detects differences in post-training data scale. For example, similarity scores decrease as the proportion of fine-tuning data increases (e.g., from 0.976 for 10% Alpaca-SFT to 0.889 for 100% Alpaca-SFT).
- Algorithm Specificity: It reveals distinct geometric signatures for different post-training strategies (PPO, DPO, RAFT). For instance, RAFT induces stronger perturbations in FFN components, while PPO causes notable drops in Q projections, whereas DPO maintains high similarity.
Baseline Comparison: Existing methods (AWM, HuReF, PDF) assign similarity scores near 1.0 for all shared-base pairs, failing to capture variations in data scale or algorithm. The proposed framework provides a continuous signal that differentiates these scenarios.

Key Contributions

Problem Formulation: Defined a hierarchical LLM lineage discrimination problem across three regimes (Independent-Origin, Same-Series, Shared-Base) with a focus on fine-grained discrimination within the shared-base setting.
Spectral Energy Insight: Demonstrated that spectral energy derived from singular values captures global magnitude patterns, enabling reliable coarse-grained separation between independent and same-series models.
Subspace Alignment Analysis: Introduced a directional geometry analysis that captures fine-grained variations in weight space, enabling discrimination among closely related shared-base models based on data scale and post-training procedures.
Empirical Validation: Validated the approach on over 110 LLM pairs, showing robustness in both regime separation and fine-grained discrimination, outperforming existing white-box and black-box baselines.

Significance and Claims

The paper claims that weight-space geometry provides a robust and interpretable signal for model lineage that does not require access to input data or training metadata.

Provenance and Governance: The method supports model provenance verification, intellectual property governance, and supply-chain transparency. It allows regulators and downstream users to identify the origin of a model and assess associated risks without needing access to proprietary training data.
Complementary Signals: The work establishes that spectral energy and subspace alignment are complementary: spectral energy handles coarse regime separation, while directional geometry handles fine-grained differentiation.
Passive and White-Box: The approach operates solely on publicly released model weights, offering a passive method for auditing the expanding ecosystem of open-weight LLMs.

The authors emphasize that their findings are based on the analysis of intrinsic structural signatures in weight space, providing a foundation for understanding how models evolve through the complex pipelines of modern LLM development.