Nano-U: Efficient Terrain Segmentation for Tiny Robot Navigation

Technical Summary: Nano-U: Efficient Terrain Segmentation for Tiny Robot Navigation

Problem Statement
Terrain segmentation is a critical perception capability for autonomous mobile robots operating in unstructured outdoor environments, such as agricultural fields and natural trails. While deep learning models have become the standard for robust segmentation, state-of-the-art architectures (e.g., U-Net, DeepLabV3) typically require hundreds of millions of parameters and GPU-based inference. These resource demands are incompatible with the strict kilobyte memory budgets and milliwatt power constraints of microcontrollers (MCUs) used in small, low-cost robotic platforms. Existing TinyML solutions often rely on dedicated ML accelerators or focus on simpler classification tasks, leaving a gap for robust, binary traversability estimation on commodity, general-purpose hardware.

Methodology
The authors propose Nano-U, a complete framework for deploying binary terrain segmentation on a low-cost ESP32-S3 microcontroller. The approach integrates three core components:

1. Network Architecture (Nano-U):

   • Inspired by the U-Net architecture, Nano-U is a highly compact encoder-decoder network designed to fit within the 320 KB internal RAM of the ESP32-S3.
   • It processes 60×80×3 color images.
   • To minimize memory usage, the network omits standard skip connections, which would require storing encoder feature maps in RAM until the decoder stage. Instead, it relies on a seven-stage pipeline using depthwise separable convolutions and nearest-neighbor upsampling.
   • The final model contains only 3,357 parameters (after layer fusion), occupying approximately 33 KB in INT8 precision.

2. Training Strategy (Quantization-Aware Distillation - QAD):

   • To compensate for the network's minimal capacity and the accuracy degradation typically associated with INT8 quantization, the authors employ a single-pass Quantization-Aware Distillation (QAD) pipeline.
   • Knowledge Distillation: A large "teacher" network (BU-Net, ~12.85M parameters) guides the student (Nano-U). The distillation loss uses temperature scaling in the sigmoid space to soften the output distribution, exposing gradient signals on ambiguous boundary pixels that hard labels would suppress.
   • Quantization-Aware Training (QAT): Fake-quantization nodes are injected from the first epoch to simulate 8-bit integer arithmetic during the forward pass while preserving full-precision gradients via the straight-through estimator.
   • The total loss function combines binary cross-entropy (hard labels) and mean squared error (soft distillation), weighted to prioritize precision for safety-critical navigation.

3. Deployment Framework (MicroFlow):

   • The quantized model is deployed using MicroFlow, a compiler-based inference engine written in Rust.
   • Unlike interpreter-based solutions (e.g., TensorFlow Lite Micro), MicroFlow resolves the network topology at compile time using procedural macros. This eliminates operator registries, dynamic memory allocation, and interpreter overhead.
   • The system runs in a no-std environment on bare metal, activating only necessary peripherals and avoiding operating system tasks (e.g., FreeRTOS), thereby maximizing available RAM and minimizing energy consumption.

Key Contributions

• Nano-U Design: A novel binary segmentation network with ~3,000 parameters that fits within the tight memory constraints of commodity MCUs while maintaining dense output capabilities.
• QAD Pipeline: A unified training regime that fuses knowledge distillation with quantization-aware training, enabling a tiny INT8 model to achieve high accuracy on challenging agricultural datasets.
• Compiler-Based Execution: An extension of the MicroFlow engine to support the specific operators required by Nano-U (MaxPool2D, nearest-neighbor upsampling), demonstrating a viable, energy-efficient execution path without dedicated ML accelerators.
• Datasets and Release: The introduction of TinyAgri, a custom dataset collected from a small rover in agricultural fields, and the public release of the Nano-U model and dataset.

Experimental Results
The framework was evaluated on two datasets: the Botanic Garden dataset (structured paths) and TinyAgri (dense vegetation, challenging lighting).

• Accuracy:

  • On Botanic Garden, the INT8 Nano-U achieved 87.0% mIoU, matching its FP32 counterpart and showing negligible degradation compared to the teacher (95.1% mIoU).
  • On the more challenging TinyAgri dataset, the INT8 model achieved 70.0% mIoU (vs. 88.3% for FP32). While boundary details were lost, the model successfully identified core traversable regions, maintaining an F1-score of 0.82.
  • The model exhibited high precision (0.96 on Botanic Garden, 0.81 on TinyAgri) with moderate recall, indicating a conservative under-segmentation strategy beneficial for safety.

• Hardware Performance (ESP32-S3-CAM):

  • Latency: Inference time was 844 ms (approx. 1.2 FPS).
  • Memory: Peak internal Data RAM usage was 257 KB, leaving ~60 KB free.
  • Power: Active inference power consumption was 470 mW (an overhead of 333 mW over the idle baseline), resulting in an energy cost of 281 mJ per inference.
  • These metrics demonstrate that the compiler-based approach significantly reduces overhead compared to standard interpreter-based deployments.

Significance and Limitations
The paper claims that this work demonstrates the feasibility of real-time binary terrain segmentation on commodity microcontrollers without specialized accelerators. The combination of QAD and compiler-based inference allows small robots to perform perception tasks previously limited to larger platforms.

The authors acknowledge specific limitations:

• Latency: At ~1.2 FPS, the system is suitable for slow-moving ground platforms but insufficient for faster vehicles or tasks requiring fine temporal resolution.
• Resolution: The 60×80 output resolution is dictated by memory constraints rather than task requirements; higher resolution would improve path planning precision.
• Generalization: The model was trained and tested on specific datasets; generalization across diverse vegetation types, lighting conditions, and soil textures requires further validation.
• Pipeline Integration: Current benchmarks isolate inference latency by loading images from PSRAM; a full closed-loop system involving live camera capture and robot control remains to be demonstrated.