Technical Summary: ARIADNE

Problem Statement

The proliferation of Parameter-Efficient Fine-Tuning (PEFT) has created model ecosystems where a single backbone language model is paired with numerous task-specialized adapters. A critical challenge arises during inference: when an input arrives without a task label, the system must automatically select the most appropriate adapter from a growing, heterogeneous pool.

Existing routing solutions face significant limitations:

1. Training-Dependent Methods: Approaches like LoRARetriever require supervised training of a dedicated routing component using labeled data and access to each adapter's training distribution, creating scalability and portability bottlenecks.
2. Spectral Routing Methods: Techniques such as Arrow and SpectR derive routing signals directly from adapter weights (e.g., via Singular Value Decomposition of LoRA matrices). While zero-shot, these methods are architecturally tied to the LoRA formulation, do not generalize to other PEFT methods, and struggle when adapters are semantically similar, often degrading to near-random accuracy.

There is a need for a routing framework that is training-free, agnostic to the specific PEFT architecture, and robust to the addition of new adapters without requiring access to adapter internals.

Methodology

ARIADNE (Agnostic Routing for Inference-time Adapter DyNamic sElection) reframes adapter routing as an input classification problem. Instead of analyzing adapter weights or gradients, it operates entirely within the latent embedding space of a frozen, off-the-shelf text encoder.

Core Mechanism

1. Centroid Construction: For each task $T_i$, the system constructs a set of $m$ representative centroids ($C_i$) in the embedding space. These are computed by clustering embeddings of samples drawn from the task's training set. Formally, for a subset of samples $S_{i,j}$, the centroid $c_{i,j}$ is the average of their embeddings:
   $$c_{i,j} = \frac{1}{|S_{i,j}|} \sum_{(x,y) \in S_{i,j}} e(x)$$
   where $e(\cdot)$ is the frozen auxiliary encoder. This multi-centroid approach captures intra-task variability better than a single global prototype.

2. Inference-Time Selection: Given an unlabeled input $x$, the system embeds it using the same frozen encoder. The routing function selects the adapter $\phi_{i^*}$ associated with the task whose centroid set yields the highest maximum cosine similarity to the input embedding:
   $$i^* = \arg \max_{i} \left( \max_{c \in C_i} \cos(e(x), c) \right)$$

Key Design Properties

• Agnosticism: Because routing relies solely on input embeddings, ARIADNE is compatible with any PEFT architecture (e.g., LoRA, VeRA, DoRA) and does not require modification to the adapters or the base model.
• Zero-Shot & Training-Free: Once centroids are computed from training data, no additional router training is required.
• Decoupling: The mechanism is decoupled from adapter internals, avoiding the need for white-box access to weight matrices.

Key Contributions

1. Reframing Routing: The authors propose framing adapter routing as an input classification problem, utilizing the latent geometry of a frozen text encoder to distinguish task distributions without relying on adapter weights.
2. Architecture Agnosticism: By operating exclusively in the input embedding space, ARIADNE is inherently compatible with any PEFT architecture, unlike spectral methods tied to LoRA.
3. Empirical Validation: The paper demonstrates that input geometry alone provides a strong signal for adapter selection, outperforming spectral routing methods and scaling robustly as the adapter library grows.
4. Failure Mode Analysis: The study reveals that routing errors are concentrated within semantically related task clusters. This leads to "graceful degradation" rather than catastrophic failure, as misrouted inputs often select adapters for similar tasks.

Results

The framework was evaluated on Llama 3.2 1B Instruct and Qwen2.5 3B Instruct backbones across diverse NLP tasks.

• Performance vs. Oracle: On a set of 23 tasks, ARIADNE achieved an average Task Performance (TP) of 54.74%, recovering 97.44% of the Oracle performance (the theoretical upper bound where the correct adapter is always selected).
• Comparison with Spectral Methods: On a shared 5-task benchmark (HellaSwag, MNLI, MRPC, QQP, SST-2), ARIADNE consistently outperformed both Arrow and SpectR. Notably, spectral methods degraded to near-random accuracy on tasks with high semantic similarity (e.g., MRPC and QQP), whereas ARIADNE maintained high accuracy.
• Scalability: In an extended study with 44 tasks, the system achieved an average Adapter Selection Accuracy (SA) of 89.7%. Performance stabilized rather than continuously degrading as the library grew, with errors remaining concentrated within semantic clusters.
• Graceful Degradation: In cases of severe routing failure (e.g., SQuAD V1 being routed to SQuAD V2), the system still recovered 85% of Oracle performance because the tasks were semantically close, validating the claim that errors result in graceful rather than catastrophic degradation.

Significance

The paper positions ARIADNE as a practical solution for dynamic adapter selection in modular LLM ecosystems. Its significance lies in its ability to provide a universal, training-free routing mechanism that scales with the growing complexity of adapter libraries without requiring privileged access to model internals. By demonstrating that input-space geometry is a sufficient and robust signal for routing, the work challenges the necessity of weight-decomposition-based methods and offers a path toward more portable and interoperable PEFT ecosystems.

The authors acknowledge a primary limitation: the method requires access to training data to compute centroids, making it less directly applicable to decentralized ecosystems where training data is proprietary. They suggest future work could explore deriving task-descriptive signals directly from adapters to achieve a fully data-free approach while maintaining agnosticism.