Design-Time Optimization of Deep Neural Networks for Intermittent Learning on Microcontrollers

Technical Summary: Design-Time Optimization of Deep Neural Networks for Intermittent Learning on Microcontrollers

Problem Statement
Deploying artificial intelligence (AI) on resource-constrained microcontroller units (MCUs) in energy-harvesting environments presents a fundamental challenge: energy availability is intermittent and non-deterministic. In scenarios such as solar-powered remote sensing, the energy buffer may deplete before a Deep Neural Network (DNN) completes an inference or training task. Traditional intermittent computing (IC) paradigms address this by saving program state to non-volatile memory (NVM) at checkpoints when energy is low, allowing execution to resume once energy is replenished. However, existing DNN design methodologies often rely on total energy consumption or accuracy as the primary metrics. This approach is insufficient for intermittent learning because a model with low total energy may still be infeasible if a single layer's energy demand exceeds the capacity of the energy buffer in one cycle. Furthermore, standard Neural Architecture Search (NAS) and Multi-Objective Optimization (MOO) approaches typically require physical deployment and repeated measurements on the target hardware to evaluate candidates, a process that is slow, error-prone, and impractical for the vast search spaces required for DNN design. Additionally, most existing energy prediction models focus solely on inference, lacking the capability to estimate the energy costs of backward passes required for on-device training.

Methodology
The authors propose a two-stage pipeline that enables offline, design-time optimization of DNNs specifically for intermittent learning on MCUs. This approach decouples the architecture search from physical deployment by utilizing a hardware-aware energy prediction model.

1. Energy Prediction Model Construction:
   The authors develop two lightweight linear regression models trained on minimal empirical data collected from a target MCU (nRF52840 Cortex-M4).

   • Layer Energy Predictor: This model estimates the energy consumption of individual DNN layers for both forward (inference) and backward (training) passes. It uses features extracted from the ONNX representation of the network, specifically the number of multiply-accumulate (MAC) operations and memory I/O bits (RAM accesses). The model accounts for hardware-specific implementation details, such as DSP instruction usage.
   • State Preservation Predictor: This model estimates the energy required to save and restore layer states to/from Ferroelectric RAM (FRAM), which is necessary for checkpointing during intermittent execution. It uses the data size (bytes) to be written or read as the primary feature.
   • Data Collection: The training dataset was generated by sampling diverse DNN architectures, compiling them to C code, and executing them layer-by-layer on the target hardware. Power profiling was performed using a Joulescope JS220, with GPIO toggling to isolate energy consumption per layer and per checkpoint operation.

2. Intermittency-Aware Multi-Objective Optimization (MOO):
   The trained prediction models are integrated into an MOO framework (using Optuna's NSGA-II sampler) to search for optimal DNN architectures.

   • Objectives: The optimization seeks to maximize validation accuracy (or minimize reconstruction loss) while minimizing total energy, memory footprint (RAM/ROM), and, crucially, the peak per-layer energy demand.
   • Constraints: The search space is constrained by the MCU's physical limits (SRAM, Flash) and the intermittent energy budget (maximum energy available per cycle).
   • Process: Candidate architectures are proposed, and their feasibility is evaluated entirely offline using the energy predictors. Only architectures that satisfy the per-layer energy budget (ensuring no single layer exceeds the buffer capacity) and memory constraints are considered. The Pareto-optimal set of solutions is then selected.

Key Contributions

• Design-Time Energy Prediction for Training: The paper introduces a method to predict per-layer energy consumption for both inference and training (backward pass) without requiring physical deployment. This extends previous work that focused primarily on inference.
• Hardware-Aware Feature Extraction: The approach utilizes a feature space based on compute (MACs) and memory access (I/O bits) derived from the ONNX model and specific hardware configurations, ensuring the predictors are portable yet accurate for the target MCU.
• Intermittency-Aware NAS: The authors demonstrate a MOO framework that explicitly incorporates intermittency constraints (specifically, per-layer energy budget limits) into the architecture search, enabling the discovery of models that are feasible under intermittent power conditions.
• Empirical Validation: The method is validated on an anomaly detection task using autoencoders on a Cortex-M4 MCU, demonstrating the ability to select architectures that significantly reduce energy consumption while maintaining performance.

Results
The evaluation was conducted using the CWRU bearing dataset for anomaly detection.

• Prediction Accuracy: The layer energy predictor achieved a Weighted Absolute Percentage Error (WAPE) of 16.6% for network layers and a Mean Absolute Percentage Error (MAPE) of 32.2%. The FRAM state preservation predictor achieved a highly accurate WAPE and MAPE of 0.9%. The authors note that while the layer error appears high, it is sufficient to reliably distinguish between feasible and infeasible architectures under strict energy constraints.
• Optimization Impact: In the MOO experiment, the approach successfully identified a Pareto front of architectures. By selecting a model with a slightly higher validation loss (Point B vs. Point A in the Pareto front), the predicted total energy demand was reduced from 0.0268 mJ to 0.00163 mJ (a reduction of approximately 94%). Crucially, the most expensive layer's energy demand dropped from 0.0198 mJ to 0.000449 mJ, ensuring the model fits within the intermittent energy buffer. Both selected models successfully separated anomalous test samples from normal data.

Significance
The paper claims to bridge the gap between Multi-Objective Optimization, automated DNN design, and deployment on energy-harvesting systems. By enabling the preemptive selection of architectures that minimize per-layer energy consumption (including checkpointing overhead), the work facilitates "truly autonomous AI at the edge." It allows for the design of DNNs that can perform on-device training in intermittent environments without the need for repeated, time-consuming physical measurements during the design phase. The authors emphasize that this approach makes it possible to deploy larger DNNs on MCUs by ensuring that no single layer exceeds the energy buffer's capacity, a constraint that traditional total-energy optimization often overlooks.