🤖 machine learning

Coverage You Can Steer: Online Conformal Calibration for RL-Driven Hardware-Aware NAS

This paper proposes an online conformal calibration framework using adaptive feedback control to restore distribution-free coverage guarantees in reinforcement learning-driven hardware-aware neural architecture search, enabling the efficient pruning of 25–50% of candidate architectures without sacrificing accuracy or violating the target error rate despite the non-exchangeable nature of the search process.

Original authors: Pedro Brandimarte, Nerea Aranjuelo, Marcos Nieto, Oihana Otaegui

Published 2026-10-05✓ Author reviewed ⓘ

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Pedro Brandimarte, Nerea Aranjuelo, Marcos Nieto, Oihana Otaegui

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where the tiny computers inside your smartwatch, your car's braking system, or a medical sensor could run the same powerful artificial intelligence that currently lives only in massive data centers. This is the promise of edge AI: intelligent systems that work locally, instantly, and without needing to send private data back to the cloud. But there is a catch. These edge devices have severe limits on how much memory they hold and how quickly they can process information. Designing a neural network—a type of computer program that learns from data—to fit these tight constraints is like trying to build a skyscraper that must also fit inside a shoebox. The number of possible designs is so vast, involving millions of combinations of layers, connections, and settings, that a human designer cannot possibly test them all.

To solve this, researchers use a method called neural architecture search, where a computer program automatically tries out different designs to find the best one. However, testing a single design is incredibly expensive and slow; it requires training the model on data to see how well it works. If a computer has to train thousands of designs just to find one good one, the process becomes too costly to be practical. The challenge, then, is to figure out which designs are worth the time and money to train, and which ones should be discarded immediately, without ever running the full test.

A team of researchers at Vicomtech in Spain has developed a new way to make this filtering process reliable. They tackled a problem that had been quietly undermining previous attempts: the computer's own learning process was changing the rules of the game while it was playing. In these search systems, a "controller" suggests new designs based on what it has learned so far. As the controller gets smarter, the designs it suggests change. Older methods for filtering out bad designs assumed that the designs suggested at the beginning of the search were statistically similar to those suggested at the end. But because the controller is learning and improving, this assumption is false. The designs keep shifting, and the old filters, which were calibrated on the early designs, began to fail. They would either waste time training terrible designs or, worse, accidentally throw away the single best design before it could be properly tested.

The researchers replaced this static, one-time filter with a system that learns and adjusts in real time. Instead of setting a rule once and hoping it holds, their new method uses a feedback loop to constantly check its own performance. After every design is tested, the system asks: "Did I correctly predict that this design would be good or bad?" If the system made a mistake, it slightly adjusts its internal threshold for what counts as a "good" design. This adjustment happens continuously, allowing the system to track the changing nature of the search. The result is a filter that can be dialed to a specific level of safety. If a researcher asks for a 90% guarantee that no good design will be missed, the system delivers exactly that, no matter how the search evolves. If they ask for 95%, it delivers that instead. This control is precise, reproducible, and works even when the designs being suggested are shifting rapidly.

The team tested this approach across three different types of network architectures and on various datasets, simulating the search for models that could run on microcontrollers with very limited memory. They found that their adaptive system could safely discard between 25% and 50% of the proposed designs without ever training them, saving a massive amount of computing time. Crucially, this pruning did not hurt the final quality of the solution. In fact, in one specific test case where the best design was a rare, isolated peak in a vast landscape of mediocre options, the old methods failed repeatedly, throwing away the best design. The new adaptive method found it every time.

The researchers also discovered that the "smart" controller often used to generate these designs was not actually the most efficient way to find the best architecture. When they compared a sophisticated learning controller against a simple random search, the random search performed just as well, if not better, in many cases. The controller was good at finding average designs, but it tended to get stuck in local areas and miss the rare, perfect ones. The true value of the new method, they found, lay not in the controller itself, but in the calibrated filter that managed the budget. By using the adaptive filter as a guide for where to look next, rather than just a gatekeeper, the researchers could steer the search directly toward the best solutions. This "calibrated optimism" allowed the system to explore promising but unproven designs with a known level of risk, outperforming random guessing and even more complex planning algorithms in constrained environments.

The study also looked at how these filters behave when the search builds a design layer by layer, rather than all at once. They found that a single, global rule for filtering often failed to account for the specific difficulties of different stages in the construction. For example, a rule that worked well for the early, shallow parts of a network might be too loose or too strict for the deep, complex parts. By applying their adaptive method separately to each stage of the construction, they could ensure that the safety guarantee held true at every step, regardless of how deep the network had grown. This granular control prevented the system from making systematic errors that would have gone unnoticed with a one-size-fits-all approach.

Ultimately, this work demonstrates that in the high-stakes game of designing artificial intelligence for limited devices, the most important tool is not necessarily a smarter predictor, but a more honest one. The old methods relied on assumptions about how the data would behave, assumptions that were broken by the very act of learning. The new method abandons those assumptions. Instead, it relies on a continuous, distribution-free check against reality, adjusting its confidence level moment by moment. It proves that you can have a search process that is both efficient and safe, one that knows exactly how much risk it is taking and can be tuned to take more or less, simply by turning a dial. This shift from static rules to dynamic, self-correcting control offers a robust path forward for building the next generation of intelligent, resource-constrained devices.

Technical Summary: Coverage You Can Steer: Online Conformal Calibration for RL-Driven Hardware-Aware NAS

Problem Statement
Hardware-aware Neural Architecture Search (HW-NAS) is constrained by the high cost of candidate evaluation, where every proposed architecture requires training to determine its reward. To mitigate this, prior work (e.g., MARCO) utilizes conformal prediction filters to prune unpromising candidates before training. These filters rely on a surrogate predictor and a conformal offset to establish an upper bound on the true reward, guaranteeing that candidates with bounds below a threshold are discarded with a controlled probability of error ( $\delta$ ).

However, standard conformal prediction assumes exchangeability between the calibration set and the test candidates. In Reinforcement Learning (RL)-driven NAS, this assumption is violated by construction:

Policy Drift: As the RL controller learns, the distribution of proposed architectures shifts away from the initial warm-up distribution used for calibration.
Within-Episode Shift: In sequential, layer-by-layer search, the distribution of partial networks changes rapidly within a single episode as depth increases.

Consequently, static conformal filters suffer from "silent degradation": their realized coverage detaches from the requested target ( $1-\delta$ ), leading to either wasted evaluations (over-coverage) or the false discarding of optimal architectures (under-coverage). Periodic recalibration mitigates variance but fails to restore control or correct depth-dependent biases.

Methodology
The paper proposes replacing static quantile estimation with Adaptive Conformal Inference (ACI) and extending it with several variants to handle the non-stationary nature of RL search.

Adaptive Conformal Inference (ACI): Instead of a fixed offset, the method employs online feedback control. After every real evaluation, the conformal offset ( $\alpha_t$ ) is updated via a subgradient step based on the miscoverage indicator:
$\alpha_{t+1} = \alpha_t + \gamma (\mathbb{I}\{r_t > g(x_t) + \alpha_t\} - \delta)$
This ensures that long-run coverage converges to the target $1-\delta$ for arbitrary candidate sequences, regardless of distribution shifts or lack of exchangeability.
AgACI (Tuning-Free): To eliminate the need for manual step-size ( $\gamma$ ) tuning, the authors aggregate multiple ACI experts (each with a different $\gamma$ ) using exponential weighting (Hedge algorithm), creating a robust, parameter-free filter.
Normalized Conformal (Locally-Adaptive): To enable exploration (acquisition), the method normalizes nonconformity scores by a per-candidate difficulty estimate ( $u(x)$ ) derived from a secondary surrogate. This yields interval widths $U(x) = g(x) + q \cdot u(x)$ , allowing the bound to rank candidates by uncertainty, not just predicted reward.
Mondrian Conformal (Group-Conditional): To address within-episode depth shifts, the method maintains separate ACI offsets for different groups (e.g., layer depth). This provides coverage guarantees conditional on the specific construction depth, correcting systematic imbalances where global offsets fail.
Conformal Acquisition: The calibrated optimistic bound is repurposed as an acquisition function. The authors implement:
- Greedy Acquisition: Selecting the candidate with the highest optimistic bound from a sampled pool.
- Conformal-Optimistic MCTS: Using the calibrated bound to value leaf nodes in a Monte-Carlo Tree Search, allowing extensive planning without real training costs.

Key Contributions

Adaptive Filtering under Policy Drift: The paper demonstrates that ACI restores "steerable coverage" in RL-driven NAS. Unlike static baselines, the adaptive filter tracks requested coverage levels ( $1-\delta$ ) to within $\sim 10^{-3}$ across three architecture families (LeNet, ResNet, MobileNet) and both single-step and sequential search formulations. It prunes 25–50% of evaluations without measurable loss in solution quality.
Conditional Extensions for Structured Search: The authors show that marginal calibration can hide systematic errors (e.g., under-coverage at deep layers). The Mondrian construction corrects this, achieving group-conditional coverage wherever the search visits a depth frequently enough to calibrate it.
From Pruning to Search: The calibrated bound is successfully used as a direct search mechanism. Greedy optimistic acquisition outperforms random search and the RL controller itself on a constrained testbed, with the online calibration providing a significant reduction in variance compared to fixed optimism.
Controller-Agnostic Analysis: The study reveals that the RL controller (faithfully reproduced from MARCO) does not outperform random search on these spaces because it optimizes expected reward rather than the argmax. The value of the proposed method lies in the calibrated bound itself, which manages the evaluation budget effectively regardless of the proposal mechanism.

Experimental Results

Calibration & Steerability: Across three datasets (MNIST, CIFAR-10, CIFAR-100), the adaptive and AgACI filters achieve monotonic, reproducible coverage. In contrast, static split-conformal baselines are unsteerable (coverage does not move with $\delta$ ) and exhibit high variance across seeds. A Gaussian Process (GP) baseline is confidently miscalibrated, with realized coverage pinned near 0.99 regardless of the requested level.
Solution Quality: On landscapes with isolated optima (ResNet-like), static filters falsely discard the best architecture on every seed due to miscalibration. Adaptive filters preserve the best solution while pruning 25–50% of candidates.
Depth-Conditional Coverage: In sequential search, global ACI under-covers at extreme depths (shallow or deep depending on the family). Mondrian adaptation restores at-target coverage for all well-sampled depths.
Search Efficiency: On a constrained testbed (256 kB memory budget), greedy conformal acquisition beats random search and the RL controller at all evaluation budgets up to 1,000. The gain is primarily driven by the optimism of the bound, with online calibration adding a final edge and reducing variance by $\sim 6\times$ .

Significance and Claims
The paper argues that inside learning loops, uncertainty must be verified against outcomes rather than assumed from a model. The primary contribution is a distribution-free, feedback-controlled calibration machinery that survives the non-stationarity of RL search.

The authors claim that:

Steerability is restored: Practitioners can dial a target risk level ( $\delta$ ) and receive it monotonically and reproducibly, a capability lost in static filters.
The controller is not the bottleneck: The value of HW-NAS methods on these spaces lies not in the learned policy (which often fails to beat random search) but in the efficient, safe management of the evaluation budget via calibrated bounds.
Acquisition is viable: The same calibrated machinery that gates evaluations can steer them, offering a distribution-free analogue to GP-UCB that does not suffer from model-based miscalibration under drift.

The paper remains modest regarding limitations, noting that the hardware cost model is analytical (not device-calibrated), results rely on partial-training proxies, and the depth-conditional guarantees degrade when specific groups (e.g., very deep layers) are data-starved. The evidence for directed search is currently limited to a single constrained testbed.

More like this