🤖 machine learning

Skill Neologisms: Towards Skill-based Continual Learning

This article proposes "skill neologisms," optimized soft-tokens integrated into an LLM's vocabulary as a scalable solution for continual learning that enables the model to acquire and combine new skills without weight updates or catastrophic forgetting.

Original authors: Antonin Berthon, Nicolas Astorga, Mihaela van der Schaar

Published 2026-05-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Antonin Berthon, Nicolas Astorga, Mihaela van der Schaar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant, well-informed librarian (the Large Language Model) who masters many tasks: sorting books, calculating numbers, writing poems, and translating languages. But what if you want to teach this librarian a brand-new trick, such as "sorting books by color," without her forgetting how to sort by author or calculate numbers?

This is the problem the work addresses. Normally, teaching a new trick involves retraining the librarian's entire brain, which carries the risk of her forgetting old skills (a problem known as "catastrophic forgetting") or simply becoming confused.

The authors propose a clever solution called "Skill Neologisms".

Here is how it works, broken down into simple concepts:

1. The Problem: The "All-or-Nothing" Trap

The Old Way (Fine-tuning): Imagine trying to teach the librarian a new skill by rewriting her entire encyclopedia. It works, but you might accidentally erase the page about "sorting by author."
The "Prompt" Way: Imagine whispering instructions to the librarian before she starts work ("Please sort by color"). This works for a specific task, but the instructions are tied to that specific job. You cannot simply combine "sorting by color" with "calculating numbers" by whispering two different things at once.

2. The Solution: "Skill Neologisms" (New Magic Words)

The authors propose giving the librarian a new vocabulary word (a "neologism") that functions like a magic switch.

The Concept: Instead of rewriting the librarian's brain, you add a new, invisible "soft token" (a special digital word) to her dictionary.
How It Is Learned: You show the librarian thousands of examples where this new word appears alongside the task she is learning (e.g., "ColorSort"). You train only this new word to understand the task, leaving the librarian's original brain completely frozen and untouched.
The Result: When you use the word "ColorSort" in a prompt, the librarian immediately knows how to perform this specific task without needing to relearn anything else.

3. The Superpower: Combining and Assembling (Composition)

The true magic happens when you combine these new words.

The Analogy: Imagine the librarian has a set of Lego bricks. Some bricks are old (skills she already knows). The authors create new, custom Lego bricks (the neologisms) for new skills.
The Claim: The work shows that if you teach the librarian two new skills separately (e.g., "ColorSort" and "SizeSort"), you can later give her a prompt with both words ("ColorSort" + "SizeSort"), and the librarian can figure out how to perform both simultaneously, even though she was never trained on this specific combination.
Why This Matters: This means you can build a library of skills. You can learn a new skill today, another tomorrow, and combine them later without ever needing to retrain the entire system.

4. What They Actually Proved

The researchers tested this idea in a controlled, "math-like" environment (not yet with real human conversations). They created a digital librarian and taught her new ways to manipulate numbers.

They Found:
- No Forgetting: The librarian did not forget old skills when learning new ones.
- Combination Works: The librarian could combine a newly learned skill with an old skill she had never seen combined before.
- Zero-Shot Combination: They could teach Skill A and Skill B separately and then combine them immediately without additional training.
- The "Sweet Spot": They discovered that the new "magic words" should not be too long or too complex. If the word is too large, the librarian becomes confused and forgets how to combine it with other skills. A small, focused word works best.

5. The Catch (Limitations)

The work is honest about what this does not yet achieve:

It Is Still a Prototype: They tested this on a small, synthetic mathematical task, not on writing novels or diagnosing diseases.
You Need the Right Data: To teach a new skill, you need a dataset where every single example requires that specific skill. If the data is messy, the new "magic word" will not learn correctly.
It Is Not Free: Although it saves storage space compared to rewriting the entire brain, training these new words still requires significant computational power.

Summary

Think of Skill Neologisms as adding new, specialized "remote control buttons" to a television. Instead of rebuilding the TV to add a new channel, you simply add a button that, when pressed, activates a specific set of circuits the TV already has. The best part? You can press two buttons at once, and the TV figures out how to handle both channels simultaneously. This work shows that this "button" approach works for AI, enabling it to continuously learn new skills without forgetting the old ones.

Technical Summary: Skill Neologisms: Towards Skill-Based Continuous Learning

1. Problem Statement

Modern Large Language Models (LLMs) demonstrate mastery of a broad range of skills as well as the ability to combine them flexibly. However, extending these capabilities to new skills in a scalable manner remains an open problem. Existing approaches face significant limitations:

Fine-Tuning and Parameter-Efficient Fine-Tuning (PEFT): Methods like LoRA carry the risk of catastrophic forgetting of previously mastered skills and can introduce safety risks.
Context-Based Approaches: In-Context Learning (ICL) exhibits limited expressiveness and is constrained by the model's effective context window.
Prompt Tuning: Although effective for task adaptation, learned prefixes are typically task-specific rather than skill-specific. They cannot be combined or adapted to new scenarios without retraining, thus failing to support zero-shot combinations of independently learned skills.

The article defines skill-based continuous learning as the goal of learning new, combinable skills without modifying model parameters. To be practical and scalable, this approach must satisfy three properties:

No Weight Updates: New skills are learned without altering the frozen model parameters.
Combinatorial Transferability: Learned skills must be combinable with existing skills, including those lying outside the distribution (OOD) of the training dataset.
Multi-Skill Combination: Multiple independently learned skills must be combinable in a zero-shot manner without joint training for their specific combination.

2. Methodology: Skill Neologisms

The authors propose Skill Neologisms, a method that integrates soft tokens directly into the model's vocabulary to represent specific procedural knowledge.

Core Components

Skill-Centric Training: In contrast to task-centric datasets, where samples implicitly draw upon various skills, Skill Neologisms are trained on datasets where every sample requires the target skill, mixed with other skills the model already masters. This fosters the learning of generally combinable representations.
Vocabulary-Level Integration: A Skill Neologism consists of a set of learnable soft tokens (a "skill token") added to the model's embedding matrix and vocabulary. These tokens are optimized on the skill-centric dataset while the rest of the model parameters remain frozen.
Insertion Function: The skill tokens are inserted into the prompt via a function $\phi_S$ (e.g., by replacing a keyword or appending an instruction) to activate the specific skill during inference.

Training Procedure

Initialization: A set of soft tokens is initialized (e.g., from the mean embedding of existing operation tokens) and added to the vocabulary.
Optimization: The tokens are trained using backpropagation through the frozen model on a skill-centric dataset. The loss function (e.g., cross-entropy) is minimized exclusively with respect to the token embeddings.
Evaluation: Performance is measured on in-distribution (ID) combinations (target skill + training skills) and out-of-distribution (OOD) combinations (target skill + held-out skills).

3. Main Contributions

Introduction of Skill Neologisms: The article presents Skill Neologisms as a pathway to skill-based continuous learning, motivated by empirical evidence that pre-trained LLMs naturally possess vocabulary tokens associated with procedural knowledge (e.g., the token "XOR" encodes the XOR operation better than a text description).
Demonstration of Combinatorial Properties: In controlled algorithmic settings, the authors show that Skill Neologisms satisfy the three required properties:
- They learn without weight updates.
- They successfully combine with OOD skills not seen during training.
- Independently trained Skill Neologisms can be combined zero-shot.
Ablation Analysis: The article analyzes the mechanisms behind the success of Skill Neologisms, specifically investigating the influence of token capacity (length) and the complexity of skill combinations in the training data.

4. Experimental Results

The authors evaluated the method on a synthetic dataset involving tasks for transforming digit sequences (e.g., sorting, adding, shifting) using a fine-tuned Qwen2.5-0.5B model.

Baseline Comparison: Skill Neologisms were compared against LoRA and Prompt Tuning.
- Combinatorial Transferability (Property 2): While LoRA and Prompt Tuning achieved high accuracy on ID combinations, they failed to generalize to OOD combinations. Only Skill Neologisms consistently succeeded in combining new skills with held-out skills.
- Multi-Skill Combination (Property 3): Independently trained Skill Neologisms for "SHIFT" and "INV-POL" were combined zero-shot. Skill Neologisms performed significantly better than In-Context Learning (ICL) baselines across various sequence lengths, demonstrating that the tokens capture reusable procedural knowledge rather than relying on pattern extraction from examples.
Ablation Findings:
- Token Capacity: There is a trade-off regarding token length. While greater capacity aids learning, excessive capacity (e.g., length > 20) reduced OOD generalization, suggesting that limited capacity acts as an inductive bias for combinable representations.
- Training Complexity: Training on more complex combinations (e.g., 3-skill combinations) improved generalization to OOD 2-skill and 3-skill combinations.
- Initialization: While initialization from pre-trained skill embeddings offered marginal improvements, random initialization also yielded strong OOD combination capabilities.

5. Significance and Claims

The article claims that Skill Neologisms offer a scalable pathway to skill-based continuous learning. By leveraging the model's existing combinatorial capabilities and integrating soft tokens at the vocabulary level, this approach enables the incremental addition of new skills without catastrophic forgetting or the need for joint retraining.

The authors position this work as a Proof-of-Concept within a controlled experimental setting. They acknowledge limitations, including the necessity of constructing diverse skill-centric datasets, the inherent optimization instability when training soft tokens, and the computational costs of backpropagation through the entire model during training. However, the results suggest that interventions at the vocabulary level can effectively encode procedural knowledge and enable LLMs to extend their capabilities in a controlled and combinable manner.