💬 NLP

When New Generators Arrive: Lifelong Machine-Generated Text Attribution via Ridge Feature Transfer

The paper proposes RidgeFT, a lightweight, replay-free framework that achieves lifelong machine-generated text attribution by training a task-aware encoder once and subsequently using closed-form ridge regression on class-wise sufficient statistics to effectively balance adapting to new generators while retaining knowledge of previous ones.

Original authors: Zhen Sun, Yifan Liao, Zhicong Huang, Jiaheng Wei, Cheng Hong, Yutao Yue, Xinlei He

Published 2026-06-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhen Sun, Yifan Liao, Zhicong Huang, Jiaheng Wei, Cheng Hong, Yutao Yue, Xinlei He

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out who wrote a specific piece of text. In the past, you only had to choose between "Human" or "Robot." But now, there are dozens of different AI robots (like GPT-4, Llama, Moonshot, etc.), and a new one appears every few months. Your job is to identify exactly which robot wrote the text.

The problem is that as new robots show up, you can't just throw away your old detective skills to learn the new ones, or you'll forget how to spot the old ones. This is called "catastrophic forgetting." Usually, to learn a new suspect, you'd have to re-study all the old case files (the text samples) from scratch, which is impossible because you don't have the data anymore, or it's too expensive to store.

The Solution: RidgeFT
The authors of this paper propose a clever new method called RidgeFT. Instead of trying to retrain your whole brain every time a new robot appears, they use a three-step "detective toolkit" that keeps your old knowledge safe while easily adding new suspects.

Here is how it works, using simple analogies:

1. The Frozen Lens (The Encoder)

Imagine you have a special pair of glasses (the "encoder") that you put on at the very beginning. You use these glasses to look at the first few robots and learn what makes them unique.

The Trick: Once you've learned the basics, you freeze these glasses. You never take them off or adjust the lenses again.
Why? If you keep adjusting the glasses to see new robots, the view of the old robots gets blurry, and you forget them. By freezing the glasses, the "view" of the old robots stays perfectly clear forever.

2. Cleaning the Noise (Covariance Calibration)

Even with your glasses on, the text might look messy. Maybe the text is about "cooking" vs. "math," or it's very long vs. very short. These are "nuisance" details that distract you from the actual robot's style.

The Trick: RidgeFT uses a mathematical filter to wash away these distractions (like topic or length) while keeping the unique "fingerprint" of the robot.
Analogy: It's like turning down the volume on the background noise in a room so you can clearly hear the specific voice of the person speaking, without needing to re-listen to old recordings.

3. The Magic Map (Random Feature Lifting)

Sometimes, the differences between robots are subtle. To make them easier to spot, RidgeFT projects the text into a new, higher-dimensional "map" using a fixed, random pattern.

The Trick: This map is like a pre-drawn grid that doesn't change. It spreads the different robots out so they don't overlap.
Why? Because the map is fixed and random, it doesn't need to be relearned. It works for both the old robots and any new ones that arrive later.

4. The Quick Update (Analytic Ridge Regression)

This is the most important part. When a brand-new robot arrives, you don't retrain your whole system.

The Trick: You only take a few "stats" from the new robot's text (like an average score and a count) and plug them into a simple math formula.
Analogy: Imagine you have a ledger (a notebook) where you write down the "average style" of each robot. When a new robot arrives, you just write down their average style in the next line. You don't need to look at their old diary entries; you just need the summary.
The Result: You instantly update your list of suspects without ever needing to see the old text again.

Why is this a big deal?

No Replaying: Most other methods try to memorize old text samples to avoid forgetting. RidgeFT doesn't need to store any old text at all. It only stores tiny summaries (statistics).
Balanced: It's great at recognizing the new robots without forgetting the old ones.
Efficient: It's very fast and doesn't require massive computer power to update.

The Bottom Line

The paper shows that by freezing the main "brain" (the encoder) and only doing simple math updates on the "summary notes" (statistics), you can build a system that learns new AI generators forever without losing its memory of the old ones. It's a simple, lightweight way to keep your detective skills sharp in a world where new AI suspects are constantly arriving.

Technical Summary: RidgeFT for Lifelong Machine-Generated Text Attribution

1. Problem Definition

The paper addresses Lifelong Machine-Generated Text (MGT) Attribution, a setting where an attribution model must continuously identify the specific generator responsible for a text as new Large Language Models (LLMs) emerge over time. Unlike static attribution tasks with a fixed set of generators, this setting is dynamic and open-ended.

The core challenge is the trade-off between adaptation (learning new generator classes) and retention (preserving the ability to recognize previously seen generators). Existing methods often fail to balance these objectives:

Full retraining is impractical due to computational costs, data licensing constraints, and the unavailability of historical data.
Continual learning via fine-tuning often leads to catastrophic forgetting, where updating the model for new classes destabilizes the decision boundaries of old classes.
Exemplar-based replay (storing past raw texts) raises privacy concerns and storage overheads.

The goal is to develop a framework that incorporates new generators efficiently without replaying historical data or continuously updating the deep text encoder, thereby maintaining a stable representation space.

2. Methodology: RidgeFT

The authors propose RidgeFT, a lightweight, exemplar-free analytic update framework. Instead of continuously fine-tuning the deep encoder, RidgeFT freezes a task-aware encoder trained on the initial generator set and performs incremental updates using closed-form ridge regression based on compact sufficient statistics.

The pipeline consists of three sequential stages:

A. Covariance Calibration
To mitigate generator-irrelevant variations (e.g., topic, domain, text length) that act as high-variance noise in the feature space, RidgeFT applies a fractional whitening transformation.

It computes a within-class scatter matrix ( $S_w$ ) from the initial training data.
It applies trace-scaled shrinkage to ensure stability in high-dimensional covariance estimation.
It performs a fractional eigenvalue decomposition to attenuate dominant noise directions while preserving the original discriminative geometry.
This transform is fixed after initialization, ensuring that incremental updates do not require revisiting old raw texts.

B. Isotropic Random Feature Lifting
To enhance the expressive capacity of the analytic classifier without introducing learnable parameters that could shift the representation space, RidgeFT maps the calibrated features into a fixed nonlinear random feature space.

A non-trainable Gaussian random matrix ( $R$ ) is sampled once before incremental learning begins.
The calibrated features are projected through this matrix, passed through ReLU, and normalized (Layer Normalization).
This isotropic projection provides a task-agnostic basis that expands the representation space uniformly, avoiding the bias of projections fitted only to initial classes.

C. Class-Balanced Analytic Ridge Regression
The classification head is updated via closed-form ridge regression using only second-order sufficient statistics.

For each class, the system stores three statistics in the random feature space: the second-order statistic ( $A_c = \sum z_i z_i^\top$ ), the first-order statistic ( $q_c = \sum z_i$ ), and the sample count ( $N_c$ ).
To address the imbalance caused by varying sample sizes (where larger classes dominate the ridge solution), the method introduces inverse-frequency weights ( $\omega_c$ ) based on class counts.
The final classifier weights ( $W$ ) are computed analytically: $W = (\bar{A} + \lambda I)^{-1} \bar{B}$ , where $\bar{A}$ and $\bar{B}$ are the weighted sums of the sufficient statistics.
When a new generator arrives, the system computes statistics for the new class, updates the weighted sums, and solves the linear system. No raw text from old classes is ever replayed.

3. Key Contributions

Problem Formulation: The paper identifies lifelong MGT attribution as a generator-evolving problem requiring the suppression of nuisance variations (topic, domain) while preserving generator-specific decision boundaries under an exemplar-free constraint.
RidgeFT Framework: It introduces an analytic update framework combining fractional covariance calibration, isotropic random feature lifting, and class-balanced closed-form ridge regression. This allows incremental updates using only compact class-wise sufficient statistics.
Empirical Validation: Extensive experiments across multiple topics (Academic, Social Media), target generators, and backbone models (DeBERTa, RoBERTa) demonstrate that RidgeFT outperforms state-of-the-art continual learning baselines.

4. Experimental Results

The authors evaluated RidgeFT using protocols P3, P4, and P5 (starting with 3, 4, or 5 initial classes and incrementally adding new ones) on datasets including MGT-Academic and AIGTBench.

Performance: Under the standard P5 protocol, RidgeFT achieved a macro-F1 of 0.886, outperforming the strongest continual-learning baseline by 0.037 in full-F1.
Retention vs. Adaptation: RidgeFT achieved an old-class F1 of 0.902 and a new-class F1 of 0.804. Notably, it improved new-class F1 by 0.107 over the best baseline, indicating superior adaptation to new generators while maintaining strong retention of old ones.
Robustness: The method remained stable even when the proportion of new-class data was reduced to 5%, demonstrating high data efficiency.
Ablation Studies: All three components (Whitening, Random Feature Lifting, Class-Balanced Ridge) were shown to be complementary. The Class-Balanced Ridge component was particularly effective in mitigating bias between old and new classes.
Storage Efficiency: While not storage-free, RidgeFT allows for significant compression. Using merged statistics in bf16 precision reduced storage requirements by 16.9× (to ~46 MiB) with negligible performance loss.

5. Significance and Claims

The paper claims that feature-stable analytic updates provide a simple yet effective approach to lifelong MGT attribution. By decoupling the learning of new generators from deep representation updating, RidgeFT avoids the representation drift associated with fine-tuning and the memory overhead of exemplar replay.

The authors position RidgeFT as a scalable solution for real-world deployment where attribution systems must evolve with the rapid emergence of new LLMs. They emphasize that their method offers a superior trade-off between retaining knowledge of established generators and adapting to new ones, addressing a central unresolved challenge in the field.

Limitations Acknowledged by the Authors:

Storage: The method is replay-free but not storage-free, as it requires storing covariance parameters and sufficient statistics.
Multilingual: Experiments were conducted primarily on English; extension to multilingual settings is suggested as future work.
Long-term Stability: The authors note that the fixed representation space may eventually become insufficient if future generators differ significantly from the initial training set, suggesting a need for periodic encoder adaptation in very long-term deployments.