💻 computer science

Once-For-All: A Train-Once and Select-Anytime Framework for Multimodal Instruction Tuning

The paper proposes OFA, a once-trained, transferable data selection framework that clusters multimodal instructions in a frozen CLIP space to identify informative samples, enabling efficient instruction tuning across diverse datasets and model scales without the need for recomputation.

Original authors: Mingkang Dong, Hongyi Cai, Xiwen Lei, Jie Li, Tao Zhang, Muxin Pu

Published 2026-05-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mingkang Dong, Hongyi Cai, Xiwen Lei, Jie Li, Tao Zhang, Muxin Pu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a very smart robot (a Vision-Language Model) how to understand the world by showing it millions of picture-and-question pairs. This is like trying to teach a child by reading them every book in a library. The problem is, most of those books are either boring repeats, written in confusing language, or just plain wrong. Reading the whole library takes forever, costs a fortune in electricity, and the robot still might not learn the most important lessons.

This paper introduces a new method called OFA (Once-For-All). Think of it as a super-efficient librarian who can look at a massive pile of books and instantly pick out the top 15% that are actually worth reading, without needing to read the whole pile first.

Here is how OFA works, broken down into simple steps:

1. The Problem: "Too Much Noise, Not Enough Signal"

Current methods for picking good data are like hiring a different expert for every new library. If you want to pick books for a science library, you hire a scientist. If you want to pick books for a history library, you hire a historian. If you change the robot you are teaching, you have to hire a new expert. This is slow and expensive.

2. The Solution: The "One-Time" Librarian

The authors built a universal selector (the librarian) that only needs to be trained once. Once trained, this librarian can walk into any library (dataset) and pick the best books for any robot (model), without needing to be retrained or re-hired.

3. How the Librarian Works (The Magic Trick)

The OFA framework uses a clever three-step process:

Step 1: Grouping by "Vibe" (Clustering)
Imagine taking all the picture-and-question pairs and sorting them into 20 different piles based on their "vibe" or topic, using a pre-trained AI (CLIP) that already understands images and text. This is like sorting books into genres without reading the whole text.
Step 2: The "Confidence" Test
The system trains a tiny, simple AI (the selector) to recognize which pile a book belongs to. But here is the trick: they stop training this AI very early.
- If the AI is very confident about a book ("Oh, this is definitely a 'Cat' book!"), that book is boring and common. It's redundant. The librarian throws it away.
- If the AI is confused or unsure ("Hmm, is this a 'Cat' book or a 'Dog' book?"), that book is interesting, complex, and valuable. The librarian keeps it.
- The Analogy: Think of a student taking a practice test. If they get an easy question right instantly, they already know it. If they struggle with a question, that's the one they need to study to get better. OFA picks the "struggle" questions.
Step 3: The "Once-For-All" Handoff
Once this tiny AI is trained, it is frozen (locked). You can now take this frozen AI and use it on a completely different library of books or a different robot. It doesn't need to learn anything new; it just applies its "confused = valuable" rule.

4. The Results: Less Data, Better Results

The paper tested this on two massive datasets (LLaVA-665K and Vision-Flan-186K).

On the training dataset: By using only 15% of the data (the "confused" samples), OFA achieved 98.3% of the performance of training on 100% of the data. It saved huge amounts of time and money.
On a new, unseen dataset: The librarian trained on the first dataset was applied to a totally different dataset (Vision-Flan) without any retraining. Surprisingly, the robot trained on this 15% subset actually performed better (110.6%) than if it had been trained on the full dataset! This proves the "confused = valuable" rule works everywhere.
On a different robot: They took the data selected by OFA and used it to train a completely different type of robot (Qwen2.5-VL-3B). It worked perfectly, proving the selection isn't tied to just one specific model.

5. Why This Matters

Speed: It takes about 9 hours to select the data, compared to 73+ hours for other methods.
Cost: It saves massive amounts of computing power (GPU hours).
Reusability: You train the selector once, and you can use it forever on new data and new models.

In summary: OFA is a smart filter that finds the "hard but useful" examples in a massive pile of data. It learns this trick once, and then can apply it to any future data or robot, saving time, money, and energy while actually making the robots smarter.

Technical Summary: Once-For-All (OFA) Framework for Multimodal Instruction Tuning

1. Problem Statement

Multimodal instruction tuning has become the standard paradigm for adapting Vision-Language Models (VLMs) to downstream tasks. However, large-scale instruction datasets (e.g., LLaVA-665K, Vision-Flan) are often massive, redundant, and occasionally contain incorrectly annotated samples. While data selection methods exist to improve training efficiency, current approaches suffer from two critical limitations:

Computational Cost: Gradient-based methods require per-sample gradient computation over the entire candidate pool, which is prohibitively expensive.
Lack of Reusability: Scoring-based and complexity-driven methods are tightly coupled to the specific target model or dataset used during signal generation. Consequently, whenever the target model architecture or the candidate dataset changes, the selection criteria must be recomputed from scratch, often at a cost comparable to training itself.

This creates a bottleneck where data selection is impractical in dynamic real-world settings where datasets and models frequently change.

2. Methodology: The Once-For-All (OFA) Framework

OFA proposes a "train-once, select-anytime" framework that decouples the data selection process from the target VLM. The core insight is that a reusable selector can be trained on a frozen representation space to identify informative samples based on their inherent difficulty or ambiguity, independent of the model being fine-tuned.

The framework operates in three main stages:

A. Multimodal Representation and Clustering

Frozen Encoding: All image-instruction pairs are encoded using a frozen CLIP model (ViT-B/32). The image and text are encoded separately and concatenated to form a joint multimodal feature vector, which is then $\ell_2$ -normalized.
Clustering: K-Means clustering is applied to these frozen features to partition the data into $K$ semantic clusters. This exposes the underlying semantic structure of the dataset without involving any target VLM.

B. Cluster-Based Pseudo-Labeling and Selector Training

Core Set Construction: A "core set" ( $D_{core}$ ) is formed by selecting samples within a specific radius ( $\gamma$ ) of their cluster centroids. These samples are assigned pseudo-labels corresponding to their cluster indices.
Lightweight Selector: A lightweight neural network (the selector) is trained on $D_{core}$ to predict the cluster pseudo-labels. Crucially, this selector is trained for only a few epochs (e.g., 3) and is deliberately early-stopped before convergence.
The Signal: The authors hypothesize that a partially trained selector will quickly become confident on "typical" (redundant/easy) samples but remain uncertain on "atypical" (informative/difficult) samples.

C. Low-Confidence Selection

Scoring: The trained selector is frozen and applied to the entire candidate dataset. For each sample, the selector outputs a confidence score defined as the maximum class probability ( $F(x_i) = \max_c p_{i,c}$ ).
Filtering: Samples with low confidence (below a per-cluster threshold $\tau_k$ ) are retained as the informative subset, while high-confidence (redundant) samples are discarded.
Transferability: Because the selector relies solely on the frozen CLIP representation and not on the target VLM's gradients or parameters, the same frozen selector can be applied to:
1. Unseen Datasets: Datasets not present during the selector's training.
2. Unseen Architectures: Target models of different scales and architectures.

3. Key Contributions

OFA Framework: A data selection framework that trains a reusable selector once on a frozen representation space, eliminating the need for recomputation when datasets or target models change.
Transferable Signal: Identification of low selector confidence on frozen CLIP representations as a generalizable signal for sample informativeness. This signal depends only on the data distribution, not the target model.
Empirical Validation: Demonstration that a single selected subset, derived from one dataset (LLaVA-665K), effectively fine-tunes both the source dataset and unseen datasets (Vision-Flan-186K) and different architectures (Qwen2.5-VL-3B, LLaVA-v1.5-7B) without re-selection.

4. Experimental Results

Experiments were conducted on LLaVA-665K and Vision-Flan-186K, fine-tuning VLMs on 15% of the data compared to full-data baselines.

Performance on Source Dataset (LLaVA-665K):
- OFA achieved 98.3% of the full-data performance across 10 benchmarks, outperforming state-of-the-art baselines like PreSel, CoIDO, and XMAS.
- Notably, OFA surpassed the full-data model on hallucination benchmarks (POPE: 87.5 vs. 86.4), suggesting the selected low-confidence samples effectively suppress hallucinations.
Cross-Dataset Transfer (Vision-Flan-186K):
- A selector trained only on LLaVA-665K was applied directly to Vision-Flan-186K without retraining.
- OFA achieved 110.6% relative performance compared to full-data training on Vision-Flan, outperforming all baselines and the full-data model itself. This indicates the presence of significant noise/redundancy in Vision-Flan that OFA successfully filtered.
Cross-Architecture Transfer:
- A subset selected from LLaVA-665K was used to fine-tune Qwen2.5-VL-3B, a structurally different model.
- OFA achieved 102.1% relative performance, surpassing both random selection and full-data training for this architecture.
Efficiency:
- OFA reduced total computational cost significantly. While full fine-tuning required 76.0 GPU-hours, OFA (including selection and fine-tuning) required only 20.0 GPU-hours.
- The selection process itself took only 9.0 GPU-hours, significantly faster than methods like Self-Filter (73.5 GPU-hours) and COINCIDE (55.5 GPU-hours).

5. Significance and Claims

The paper claims that OFA addresses a fundamental practical limitation in multimodal instruction tuning: the non-reusability of current data selection signals. By decoupling the selection process from the target model, OFA provides a single, reusable solution that is effective across diverse datasets and model architectures.

The authors emphasize that the "low-confidence" signal derived from a deliberately under-trained selector on frozen CLIP features generalizes to data distributions and architectures never seen during training. This "Once-For-All" approach transforms data selection from a costly, per-task computation into a one-time investment, enabling efficient and robust training of VLMs in resource-constrained or dynamic environments.