A Constant-Time Implementation Methodology for Activation Functions on Microcontrollers

Technical Summary: Constant-Time Implementation Methodology for Activation Functions on Microcontrollers

Problem Statement
Embedded neural-network inference on resource-constrained microcontrollers is increasingly vulnerable to timing side-channel attacks. Recent studies have demonstrated that activation functions within these networks leak information through execution-time variations dependent on input values and function types. This leakage allows attackers to distinguish between different activation functions (e.g., ReLU vs. sigmoid) and potentially reverse-engineer model architectures or infer sensitive input data. While existing countermeasures, such as desynchronization (introducing random delays), attempt to obscure these timing patterns, they do not eliminate the underlying input-dependent behavior. Consequently, profiled template-based attacks can still statistically distinguish activation functions despite the added noise.

Methodology
The authors propose a constant-time implementation methodology designed to eliminate input-dependent timing variations at the source, rather than merely hiding them. The approach is instantiated on an ARM Cortex-M4 platform (STM32F411) and targets five widely used activation functions: ReLU, sigmoid, tanh, Gaussian Error Linear Unit (GELU), and Swish.

The methodology relies on four core components:

1. Branchless, Masking-Based Selection: To prevent control-flow divergence, the implementation avoids data-dependent branches. Instead, it uses logical masking and bitwise operations (AND/OR) to select between candidate values (e.g., saturated outputs vs. approximation results) based on the input's bit representation. This ensures the instruction sequence remains identical regardless of the input value.
2. Fixed-Cost Padé-Based Approximation: For nonlinear functions (sigmoid, tanh, GELU, Swish), the authors replace standard transcendental function evaluations with a shared, fixed-cost rational approximation based on the Padé approximant for $\tanh(x)$, denoted as $R_{tanh}(x)$. This approximation involves a fixed sequence of additions, multiplications, and a single division, avoiding loops or interval selection dependent on the input.
   • Sigmoid and Swish are constructed using the identity $S(x) = \frac{1}{2}(1 + \tanh(x/2))$.
   • GELU is approximated using a $\tanh$-based formulation.
3. Masking-Based Saturation: To handle inputs outside the approximation region, the methodology employs saturation logic (clamping to 0, 1, or $x$) implemented via the same branchless masking strategy. This ensures that even when the rational approximation is bypassed for saturated inputs, the execution path and cycle count remain constant.
4. Cycle Alignment and Dummy Arithmetic: To achieve identical execution times across all five functions, the authors introduce dummy arithmetic operations. Specifically, the simple ReLU function, which naturally executes much faster than the nonlinear approximations, is augmented with the same $R_{tanh}$-based arithmetic sequence (without affecting the final result) to align its cycle count with the more complex functions. Minimal NOP padding is used only for fine-grained alignment.

Key Contributions

• Constant-Time Methodology: The paper introduces a reusable construction principle for activation functions that guarantees input-independent execution time on embedded microcontrollers.
• Instantiation on Five Functions: The methodology is successfully applied to ReLU, sigmoid, tanh, GELU, and Swish, utilizing a shared rational approximation for the nonlinear components.
• Vulnerability of Desynchronization: The authors demonstrate that desynchronization-based countermeasures, while visually obscuring timing traces, remain vulnerable to profiled template-based timing attacks. Statistical analysis of execution times allows an attacker to distinguish activation functions with high confidence after sufficient measurements.
• Experimental Validation: The work provides empirical evidence that the proposed protected implementations achieve identical cycle counts for all tested inputs, effectively suppressing timing leakage.

Experimental Results
The methodology was evaluated on an ARM Cortex-M4 running at 84 MHz.

• Timing Regularity:
  • In a three-function setting (ReLU, sigmoid, tanh) over the interval $[-8, 8]$, all protected implementations executed in exactly 88 clock cycles (1.048 µs), regardless of the input value.
  • In an extended five-function setting (including GELU and Swish) over the interval $[-500, 500]$, all implementations executed in exactly 108 clock cycles (1.286 µs).
  • In contrast, unprotected implementations showed significant timing variations (e.g., unprotected ReLU took ~12 cycles, while unprotected tanh took ~403 cycles) and input-dependent fluctuations.
• Numerical Accuracy:
  • The approximations retained high accuracy. For the interval $[-8, 8]$, the Mean Squared Error (MSE) for sigmoid was $2.93 \times 10^{-12}$ and for tanh was $6.14 \times 10^{-10}$.
  • In the extended interval $[-500, 500]$, errors remained negligible for sigmoid and tanh (as they saturate quickly), while GELU and Swish showed low error localized near saturation thresholds (Max absolute error for Swish was $1.11 \times 10^{-3}$).
• Performance Trade-offs:
  • The protected ReLU implementation incurred a latency increase (from ~12 to 88/108 cycles) due to the insertion of dummy arithmetic for alignment.
  • Conversely, the protected nonlinear functions were significantly faster than their unprotected counterparts (e.g., protected sigmoid/tanh were ~33-38% faster than unprotected versions) because the rational approximation is more efficient than standard library transcendental functions on the target hardware.

Significance and Claims
The paper claims that the proposed methodology provides a practical basis for constructing side-channel-resistant activation functions in embedded inference. By shifting from "hiding" (randomization) to "constant-time" design, the approach eliminates the root cause of timing leakage: input-dependent execution paths. The authors assert that this methodology allows for the construction of activation functions where the executed instruction sequence is independent of the input value, thereby preventing template-based timing attacks.

The work positions itself as a building block for secure embedded machine learning, particularly for privacy-sensitive edge devices (e.g., medical sensors, wearables) where timing information may be accessible to attackers. The authors modestly note that while this addresses timing leakage in activation functions, it does not constitute a complete countermeasure for the entire inference pipeline, and other side channels (power, EM) or implementation details may still require protection. The study focuses specifically on the timing domain and does not claim to address all potential side-channel vectors.