🤖 machine learning

Feasible-First Exploration for Constrained ML Deployment Optimization in Crash-Prone Hierarchical Search Spaces

This paper proposes Thermal Budget Annealing (TBA), a feasible-first exploration method that combines early trial timeouts and subspace blacklisting with warm-started Tree-structured Parzen Estimators to efficiently optimize constrained machine learning deployments in crash-prone hierarchical search spaces, validated by the new DeployBench benchmark across diverse GPU targets.

Original authors: Christian Lysenstøen

Published 2026-04-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Christian Lysenstøen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef trying to create the perfect dish, but you have a very strict rule: you only have 25 ingredients to test before you run out of money.

Your kitchen is chaotic. Some ingredient combinations explode (crash), some take hours to cook (too slow), and some simply don't work together (incompatible). You need to find the best tasting dish that fits within your time and budget limits.

This is exactly the problem the paper solves, but instead of a kitchen, it's about deploying Machine Learning models onto computer chips (GPUs).

Here is the breakdown of the paper's story, using simple analogies:

1. The Problem: The "Crash-Prone" Kitchen

In the past, computer scientists used smart algorithms (like TPE) to find the best settings for AI models. These algorithms are like a chef who tastes a few dishes, learns what flavors work, and then starts guessing the next dish based on what they learned.

But there's a catch: In the real world of AI deployment, most random guesses are disasters.

You pick a model and a setting, and the computer runs out of memory (it "crashes").
You pick a setting, and it takes 5 minutes to run when you only have 20 seconds.
You pick a setting, and the software refuses to run it.

If your "smart chef" (the TPE algorithm) spends its first 10 tries on exploding dishes or dishes that take forever to cook, it runs out of ingredients before it ever finds the best type of dish. It gets stuck optimizing a "good enough" dish (like a standard burger) because it never got a chance to taste the "perfect" dish (a rare, complex recipe) that it missed early on.

The paper calls this "Premature Exploitation." The chef stops exploring too early and starts perfecting the wrong thing.

2. The Solution: "Thermal Budget Annealing" (TBA)

The authors propose a new two-step strategy called TBA → TPE. Think of it as a two-phase cooking contest:

Phase 1: The "Feasible-First" Scouting Mission
Before the smart chef starts guessing, they send out a scout with a specific job: Find anything that doesn't explode.

The scout uses a method called Simulated Annealing. Imagine this as a "wild exploration" mode where the chef tries every type of main ingredient (Model Family) just to see which ones can actually be cooked without blowing up the kitchen.
The Safety Nets:
- Trial Timeouts: If a dish starts taking way too long to cook (e.g., 5 minutes when the limit is 20 seconds), the scout pulls the plug immediately. They don't wait for it to finish; they just mark it as "too slow" and move on.
- Subspace Blacklisting: If the scout tries "Spicy Salsa" three times in a row and it explodes every time, they put "Spicy Salsa" on a temporary "Do Not Touch" list. They stop wasting time on it for a while, but they don't ban it forever (in case it works with a different main ingredient).

Phase 2: The Smart Chef Returns
Once the scout has found a list of "safe" ingredients and configurations, they hand that list to the Smart Chef (TPE).

Now, the Smart Chef doesn't have to guess blindly. They start with a "warm start"—a head start based on the scout's map of what works.
Because the scout already explored the dangerous zones, the Smart Chef can focus on fine-tuning the best options without wasting time on explosions.

3. The Results: Finding the "Vit-Tiny" Treasure

The researchers tested this on five different computer chips (GPUs), ranging from powerful data-center servers to smaller laptop chips.

The Old Way (Cold-Start TPE): Often got stuck. On the RTX 5080 laptop chip, the old method found the best model (called vit_tiny) in only 3 out of 10 attempts. It kept picking a "safe but mediocre" model (resnet50) because it never got a chance to try the best one.
The New Way (TBA → TPE): Found the best model (vit_tiny) in 8 out of 10 attempts on the same chip.
The Efficiency: The new method wasted less "ingredients" (budget) on failed attempts. While random guessing found the best model often, it wasted 74% of its budget on crashes. The new method wasted only 42%.

4. The Core Lesson

The paper's main conclusion is simple but powerful: In a dangerous, crash-prone environment, you cannot rely on a smart algorithm to figure out the basics.

If you let a smart algorithm start immediately, it might get trapped in a small corner of the search space because it ran out of time exploring the rest. You need a dedicated "scouting phase" to map out the safe zones first.

The Analogy Summary:

The Problem: Trying to find the best route through a city where 50% of the streets are blocked or lead to dead ends.
The Old Method: A GPS that tries to calculate the fastest route immediately. It gets stuck in a small neighborhood because it ran out of battery trying to navigate the blocked streets.
The New Method: A drone flies over the city first (Phase 1) to mark which streets are open. Then, the GPS (Phase 2) uses that map to find the fastest route. The GPS finds the destination much more often and uses less battery.

The paper proves that for AI deployment, exploring first, then exploiting, is the key to success when the budget is tight and the environment is hostile.

1. Problem Statement

The paper addresses the challenge of optimizing Machine Learning (ML) model deployment under strict production constraints (latency, memory, power). Unlike standard hyperparameter optimization (HPO), deployment optimization presents three unique challenges:

Hierarchical Search Spaces: The validity of parameters depends on structural choices (e.g., selecting a specific backend activates different compiler options).
Partial Invalidity (Crash Zones): A significant portion of configurations (30–80%) are invalid due to out-of-memory (OOM) errors, unsupported operator combinations, or compiler failures. These configurations do not return an objective value but rather a crash.
Unequal Evaluation Costs: Evaluating a configuration can range from seconds to minutes. Wasting budget on crashing configurations is operationally expensive.

The Core Failure Mode: Standard model-guided optimizers (like Tree-structured Parzen Estimators, TPE) rely on early samples to build a surrogate model. In "crash-heavy" environments, these optimizers often spend their limited budget on invalid trials or converge prematurely to a suboptimal model family (e.g., ResNet-50) because they never observe the globally optimal family (e.g., ViT-Tiny) before the exploitation phase begins. This is termed premature exploitation.

2. Methodology: Thermal Budget Annealing (TBA)

The authors propose TBA, a hybrid optimization strategy that decomposes the process into two distinct phases: Feasible-First Exploration followed by Model-Guided Exploitation.

Phase 1: Feasible-First Simulated Annealing (SA)

Instead of immediately fitting a probabilistic model, Phase 1 uses a modified Simulated Annealing algorithm to map valid regions and ensure structural diversity.

Two-Stage Objective:
- Feasibility Stage: Before finding any valid configuration, the algorithm minimizes total constraint violation ( $V(x)$ ).
- Optimization Stage: Once a feasible configuration is found, the objective switches to maximizing the deployment metric (e.g., accuracy).
Adaptive Temperature: Uses a reheating schedule to maintain broad exploration during the feasibility stage.
Hierarchical Neighbor Proposals: The algorithm explicitly performs "structural moves" (changing model family, backend, or quantization) with high probability early on, decaying over time to favor local refinement.
Robustness Mechanisms for Hostile Hardware:
- Trial Timeouts: Aborts trials early if initial latency measurements exceed $5\times$ the constraint threshold, preventing wasted time on catastrophically infeasible configurations.
- Subspace Blacklisting: Temporarily suppresses specific categorical values (e.g., a specific quantization mode) after three consecutive failures, preventing the search from getting stuck in "crash traps."
- Elite Restart & Snap-back: Mechanisms to recover from local minima and avoid stagnation.

Phase 2: Warm-Started TPE

Once the exploration phase gathers sufficient data (minimum feasible and invalid observations), the entire history (including crashes and constraint metadata) is injected into Optuna's TPE sampler.

Adaptive Handoff: The transition occurs when specific criteria are met (e.g., $N_{feas} \ge 5$ , $N_{bad} \ge 3$ ), ensuring the TPE starts with a diverse, informative dataset rather than a cold start.
Constrained Acquisition: TPE uses the collected history to guide exploitation, now conditioned on a broader understanding of the feasible space.

3. Key Contributions

Problem Formalization: Defined budget-constrained deployment optimization in partially invalid hierarchical spaces, identifying "premature exploitation" as a critical failure mode.
TBA Algorithm: Introduced a feasible-first exploration procedure with adaptive temperature, crash-aware state transitions, and specific robustness mechanisms (timeouts, blacklisting) tailored for hostile hardware.
DeployBench: Created a new benchmark suite featuring hidden crash zones, hierarchical conditionals, and unequal evaluation costs, simulating real-world deployment scenarios across five GPU targets (H100, A100, RTX 5080, L4, T4).
Empirical Validation: Demonstrated that warm-starting TPE with feasible-first exploration significantly improves the discovery of the globally optimal model family under tight constraints compared to cold-start TPE.

4. Experimental Results

The method was evaluated on synthetic benchmarks (Crashy Branin) and real-world ImageNette deployment across five GPU hardware targets.

Synthetic Benchmarks: TBA→TPE outperformed cold-start TPE and Random Search in finding the best feasible objective, particularly at low budgets ( $B=20$ ). It reduced the "wasted budget fraction" (trials that crashed or violated constraints) significantly.
Real Deployment (RTX 5080):
- Discovery Rate: The hybrid method discovered the globally optimal model family (vit_tiny) in 8/10 seeds, whereas cold-start TPE only found it in 3/10 seeds.
- Efficiency: While Random Search also found vit_tiny frequently (8/10), it wasted 74% of its budget on invalid trials. The hybrid wasted only 42%.
- Premature Exploitation: TPE frequently converged to resnet50 (a suboptimal choice) in 7/10 seeds because its early random samples were biased toward that family, and it never explored vit_tiny later.
Multi-Hardware Generalization:
- The hybrid matched or exceeded TPE's discovery rate across all five GPUs (H100, A100, RTX 5080, L4, T4).
- H100: Hybrid found vit_tiny in 5/5 seeds vs. TPE's 3/5.
- T4 (Hostile): With robustness mechanisms, the hybrid matched TPE (3/5) but with lower waste (36% vs. 38%). Without these mechanisms, the hybrid failed on T4.
Trade-off: The hybrid occupies a unique position in the "Discovery vs. Waste" trade-off: it achieves high discovery rates comparable to Random Search but with significantly lower wasted budget, outperforming TPE which has low waste but poor discovery in crash-heavy regimes.

5. Significance and Conclusion

The paper argues that in constrained, crash-prone deployment environments, the quality of early exploration determines the success of later exploitation.

Narrow but Critical Claim: The authors do not claim TBA is a universal replacement for TPE. Instead, they prove that in hostile spaces with small budgets, explicit feasible-first exploration is necessary to prevent model-guided optimizers from locking onto suboptimal model families.
Practical Impact: By reducing wasted evaluation time and ensuring the discovery of the best model architecture (e.g., vit_tiny), this approach offers a more reliable and cost-effective path for deploying ML models on diverse hardware.
Future Work: The authors suggest adaptive handoff criteria based on structural diversity and extending the benchmark to NLP and multi-model pipelines.

In summary, TBA successfully bridges the gap between robust exploration in hostile environments and efficient exploitation, solving the "premature exploitation" problem that plagues standard AutoML tools in real-world deployment scenarios.