OASI: Objective-Aware Surrogate Initialization for Multi-Objective Bayesian Optimization in TinyML Keyword Spotting

This paper proposes Objective-Aware Surrogate Initialization (OASI), a method that seeds multi-objective Bayesian optimization with Pareto-biased solutions to efficiently discover memory-feasible, high-accuracy keyword spotting models for resource-constrained TinyML hardware.

The Gist

Imagine you are trying to build a tiny, super-smart robot that lives inside a small, battery-powered device (like a smart speaker or a hearing aid). This robot's only job is to listen for a specific "wake word" (like "Hey Siri" or "Alexa") and wake up the bigger system when it hears it.

The problem is that this robot has to fit into a very small box with very little memory (RAM) and very little battery. If the robot is too smart but too big, it won't fit in the box. If it's small enough but too dumb, it won't understand the word. You need the perfect balance: Maximum Smarts + Minimum Size.

This is exactly what the paper "OASI" is about. Here is the breakdown using simple analogies.

1. The Problem: The "Tiny Box" Dilemma

Think of the microcontroller (the chip in the device) as a tiny backpack.

• Accuracy is how many heavy books (knowledge) you want to carry.
• Memory (RAM/Flash) is the size of the backpack.
• Energy is how tired you get carrying it.

You want to carry the most useful books possible, but the backpack has a strict weight limit. If you try to stuff too many books in, the backpack rips (the device crashes). If you carry too few, you can't find your way (the device makes mistakes).

2. The Old Way: "Throwing Darts Blindfolded"

To find the perfect backpack setup, engineers usually use a method called Bayesian Optimization. Think of this as a smart explorer trying to find the best spot on a map.

However, the explorer needs to start somewhere. The old methods (called LHS, Sobol, or Random) are like throwing darts blindfolded at a giant wall to find the "good" spots.

• They pick random spots.
• Many of those spots are terrible (e.g., a backpack that is too heavy and rips immediately).
• Because they start with so many bad guesses, the explorer wastes time and energy before finding the "sweet spot" where the backpack is just right.

In the world of TinyML, you don't have much time or battery to waste on bad guesses.

3. The New Solution: OASI (The "Smart Scout")

The authors propose a new method called OASI (Objective-Aware Surrogate Initialization).

Instead of throwing darts blindfolded, OASI sends out a Smart Scout first.

• The Scout's Job: Before the main explorer starts, the Scout runs a quick, rough simulation (using a technique called "Simulated Annealing").
• The Scout's Strategy: The Scout doesn't look at random spots. It specifically looks for backpacks that are already a good balance of size and smarts. It finds the "Pareto-optimal" spots—places where you can't get smarter without getting bigger, or smaller without getting dumber.
• The Handoff: The Scout then hands these "pre-vetted" good spots to the main explorer.

The Analogy:

• Old Way: You walk into a massive library and start reading random books to find the best one. You might spend hours reading trashy novels before finding a classic.
• OASI Way: A librarian (the Scout) runs ahead, finds the top 10 best books based on your taste, and places them on a table for you. You start your search right there. You find the perfect book much faster.

4. Why This Matters (The Results)

The paper tested this on real hardware (STM32 microcontrollers, which are the brains of many small devices).

• The "Random" explorers often picked backpacks that were too heavy. When they tried to put them on the real device, the device crashed (Out of Memory errors).
• The "OASI" explorer started with backpacks that were already known to fit.
  • It found a better balance of Smarts vs. Size.
  • It converged (found the answer) faster.
  • Most importantly, the models it picked actually worked on the real devices without crashing.

5. The "Deployability Index"

The authors also created a score called the Deployability Index (DI).
Think of this as a "Fit Score."

• A score of 1.0 means the backpack fits perfectly with room to spare.
• A score of 0 means the backpack is too big and won't fit.
• OASI consistently got high Fit Scores, meaning the robots it designed were ready to be shipped and used immediately.

Summary

OASI is a "smart start" technique. Instead of guessing randomly where to build a tiny AI model, it uses a quick preliminary search to find the "golden zone" of performance and size first. This saves time, prevents the AI from being too big for its hardware, and ensures the final product actually works in the real world.

It's the difference between blindly searching for a needle in a haystack and having a magnet that points you straight to the needle.

Technical Summary

Here is a detailed technical summary of the paper "OASI: Objective-Aware Surrogate Initialization for Multi-Objective Bayesian Optimization in TinyML Keyword Spotting."

1. Problem Definition

The paper addresses the challenge of designing Keyword Spotting (KWS) models for TinyML applications (microcontroller-class hardware). The core problem is a constrained Multi-Objective Optimization (MOO) task where the goal is to simultaneously:

• Maximize recognition accuracy.
• Minimize model size (Flash usage) and runtime memory (SRAM).

Key Constraints & Challenges:

• Strict Resource Limits: Microcontrollers have very limited Flash and SRAM (often < 2 MB). A model failing to fit within the SRAM budget during inference is unusable, regardless of its accuracy.
• Evaluation Cost: Training and validating neural networks for every candidate architecture is computationally expensive.
• Initialization Sensitivity: Multi-Objective Bayesian Optimization (MOBO) is highly sensitive to its initial dataset. Standard initialization methods (e.g., Latin Hypercube Sampling, Sobol sequences, or random sampling) are "objective-agnostic." They sample the search space uniformly but often generate infeasible configurations (e.g., models that exceed SRAM limits) or fail to identify the Pareto-optimal trade-offs early in the optimization process, leading to slow convergence or wasted evaluation budgets.

2. Methodology: OASI (Objective-Aware Surrogate Initialization)

The authors propose OASI, a novel initialization strategy designed to seed MOBO with high-quality, Pareto-biased solutions before the Bayesian optimization loop begins.

Core Components:

• Multi-Objective Simulated Annealing (MOSA): Before the MOBO phase, OASI runs a short MOSA process (40–50 iterations per chain) to explore the architectural search space.
  • It generates candidate architectures by perturbing hyperparameters (depth, filters, kernel size, etc.).
  • It uses a probabilistic acceptance criterion based on both accuracy (maximization) and size (minimization) to guide the search toward the Pareto front.
• Surrogate Conditioning: The solutions generated by MOSA are stored in an "initializer archive." A diverse subset is selected to form the initial dataset ($D_0$) for the MOBO surrogate model (Gaussian Process).
• Objective-Aware Bias: Unlike space-filling methods, OASI explicitly biases the initial data toward feasible accuracy-memory trade-offs. This ensures the Gaussian Process surrogate is "calibrated" early on to understand the constraints of the TinyML environment, avoiding the exploration of infeasible regions (e.g., SRAM violations).

Optimization Workflow:

1. OASI Phase: Generate Pareto-biased seeds via MOSA.
2. MOBO Phase: Use these seeds to train Gaussian Process surrogates for Accuracy and Flash size.
3. Acquisition: Use Expected Hypervolume Improvement (EHVI) to select the next candidate.
4. Constraint Handling: Peak SRAM usage is treated as a hard constraint ($RAM \leq B_{ram}$). Latency is evaluated separately in a hardware-in-the-loop phase.

3. Key Contributions

1. OASI Framework: Introduction of a MOSA-based initialization strategy specifically for MOBO in TinyML, addressing the trade-off between model accuracy and size.
2. Superior Convergence: Demonstration that objective-aware initialization leads to more stable convergence and higher-quality Pareto fronts compared to standard methods (LHS, Sobol, Random) under tight evaluation budgets.
3. Hardware Validation: Successful deployment of optimized models on physical STM32 microcontrollers, proving that OASI avoids "out-of-memory" failures that plague other initialization strategies.
4. Deployability Index (DI): Proposal of a quantitative metric ($DI$) that measures deployment readiness by penalizing saturation in any single resource dimension (SRAM, Flash, Latency).

4. Experimental Results

The study was conducted on the Google Speech Commands v2 dataset using a Depthwise Separable CNN (DS-CNN) backbone.

Comparison Metrics:

• Optimizers: OASI-MOBO vs. NSGA-II, MOSA, and standard MOBO (Random init).
• Initialization Strategies: OASI vs. LHS, Sobol, and Random (within the MOBO framework).

Key Findings:

• Performance: OASI-MOBO achieved the best trade-off with 90.1% accuracy and 0.103 MB model size, significantly outperforming NSGA-II (88.9%, 0.210 MB) and standard MOBO (88.5%, 0.120 MB).
• Hypervolume (HV) & Generational Distance (GD): OASI achieved the highest Hypervolume (0.0627) and a Generational Distance of 0.0, indicating perfect convergence to the reference Pareto front. In contrast, other methods had non-zero GD and lower HV.
• Efficiency: While OASI has a slightly higher initialization overhead (~1.9s vs ~1.5s for others), it drastically reduces the total time to find a high-quality solution by avoiding uninformative early iterations.
• Hardware-in-the-Loop (HIL) Deployment:
  • Models selected by OASI were successfully compiled and deployed on STM32H7, F469, and F401 microcontrollers.
  • Critical Success: OASI-selected models fit within strict SRAM budgets (e.g., 51.0 KB peak RAM on NUCLEO-F401RE), whereas models from other strategies often failed due to memory constraints.
  • Latency: OASI models demonstrated superior latency on hardware accelerators (e.g., 0.315 ms on STM32N6570-DK with Neural-ART offloading).

5. Significance

This paper bridges the gap between theoretical multi-objective optimization and practical TinyML deployment.

• Practicality: It solves a critical real-world bottleneck: the high failure rate of automated model search due to memory constraints. By "warming up" the optimizer with feasible solutions, OASI ensures that the search remains within the deployable region.
• Resource Efficiency: It proves that spending a small amount of computational budget on intelligent initialization (MOSA) yields a massive return on investment in terms of final model quality and deployment success.
• Standardization: The introduction of the Deployability Index provides a new, rigorous way to evaluate not just model accuracy, but the actual feasibility of running that model on specific edge hardware.

In conclusion, OASI represents a significant advancement in automated machine learning for edge devices, ensuring that the search for optimal models is not just mathematically sound but also hardware-feasible.