🤖 machine learning

Ariel-ML: Computing Parallelization with Embedded Rust for Neural Networks on Heterogeneous Multi-core Microcontrollers

This paper introduces Ariel-ML, an open-source embedded Rust toolkit that automates parallel inference for TinyML models on heterogeneous multi-core microcontrollers, demonstrating superior latency performance while maintaining memory footprints comparable to traditional C/C++ solutions.

Original authors: Zhaolan Huang, Kaspar Schleiser, Gyungmin Myung, Emmanuel Baccelli

Published 2026-05-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhaolan Huang, Kaspar Schleiser, Gyungmin Myung, Emmanuel Baccelli

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a tiny, battery-powered computer (like the kind inside a smart thermostat or a fitness tracker). For years, these devices have been like solo artists: they have only one brain cell (a single processor core) to do all their thinking. They are also written in an old, slightly risky programming language called C/C++, which is like building a house with wooden beams that can rot if you aren't careful.

Now, two big changes are happening:

The Hardware Upgrade: New tiny computers are arriving with multiple brain cells (multi-core processors), like a choir instead of a soloist.
The Safety Upgrade: Developers are switching to a new language called Rust, which is like using steel beams that are much harder to break or corrupt.

The problem? Until now, there was no easy way to teach these new "choir" computers how to work together to solve complex puzzles (like recognizing a voice or identifying a cat in a photo) using the new "steel" language. Existing tools were either stuck in the old language or couldn't tell the multiple brain cells how to share the work.

Enter Ariel-ML.

Think of Ariel-ML as a super-efficient conductor for this new choir. Here is how it works, using simple analogies:

1. The Translator (The Build Pipeline)

Imagine you have a complex recipe written in a fancy chef's language (like PyTorch or TensorFlow). Ariel-ML takes this recipe and translates it into a specific set of instructions that the tiny computer understands. It uses a tool called IREE (think of it as a high-tech translation machine) to break the big recipe down into small, manageable tasks.

2. The Conductor (The Multicore Scheduler)

This is the paper's biggest innovation. Once the recipe is broken down, Ariel-ML's "conductor" looks at the multiple brain cells (cores) in the device.

Old way: One brain cell tries to do everything while the others sit idle, waiting for a turn.
Ariel-ML way: The conductor instantly hands out different parts of the recipe to different brain cells. If you have two brain cells, they split the work in half and finish the job almost twice as fast. The paper calls this a "greedy scheduler," which is just a fancy way of saying it grabs any free brain cell and immediately gives it a task, ensuring no one is ever sitting around doing nothing.

3. The Safety Net (Rust)

Because Ariel-ML is built entirely in Rust, it acts like a safety net. In the old days (C/C++), if a program tried to use memory it didn't own, the whole system could crash. Rust prevents this from happening by design. The paper claims this makes the software much more secure and reliable, which is crucial as these devices become more connected to the internet.

What Did They Prove?

The authors tested Ariel-ML on several real-world tiny computers (like the Raspberry Pi Pico and Nordic chips). They compared it to the old tools:

Speed: On computers with two brain cells, Ariel-ML was up to 1.6 times faster than the old tools. It successfully got the choir to sing in harmony.
Memory: The paper admits that because Rust and the new translation tools are so robust, the final program takes up a bit more space in the computer's memory (like a slightly heavier backpack). However, they showed that the speed gains on multi-core devices make this extra weight worth it.
Accuracy: They double-checked the math and confirmed that Ariel-ML gets the exact same answers as the big, powerful computers used to train the models.

The Bottom Line

Ariel-ML is the first toolkit that lets developers take modern, complex AI models, translate them into the safe, modern Rust language, and automatically split the work across multiple brain cells in tiny, low-power devices. It bridges the gap between the "old school" single-core devices and the "new school" multi-core future, allowing these tiny gadgets to think faster and safer without needing a massive battery upgrade.

Technical Summary: Ariel-ML

Problem Statement
The paper addresses a critical gap in the deployment of TinyML (Tiny Machine Learning) on modern, resource-constrained microcontrollers (MCUs). While the hardware landscape is shifting from single-core to predominantly multi-core architectures (e.g., Arm Cortex-M, RISC-V, ESP32), and the embedded software ecosystem is increasingly adopting Rust for its memory safety and security benefits, no existing solution bridges these trends. Specifically, there was no Rust-based embedded software platform capable of automating parallel inference computation for arbitrary TinyML models on heterogeneous multi-core MCUs. Existing Rust ML frameworks (e.g., Burn, Candle) are not tailored for low-power MCUs, and existing TinyML pipelines (e.g., RIOT-ML, U-TOE) are primarily C/C++ based and lack native multi-core parallelization support.

Methodology
The authors introduce Ariel-ML, a novel toolkit designed to fill this gap. The system combines a generic TinyML pipeline with an embedded Rust software platform (Ariel OS) to leverage multi-core capabilities. The architecture consists of two main phases:

Host-Side Build Pipeline:
- The system integrates the IREE (Intermediate Representation Execution Environment) compiler.
- It transpiles models from mainstream frameworks (TensorFlow, PyTorch, ONNX) into IREE modules.
- During compilation, IREE partitions operator computations (e.g., convolution, matrix multiplication) into contention-free "work items" (tiles).
- Metadata regarding core counts and entry points is constructed and co-compiled with the application code, IREE runtime, and Ariel OS to produce a firmware image.
Device-Side Runtime:
- Ariel-ML Core: Written in Rust, this component binds to the IREE runtime and includes a greedy multicore scheduler. It dispatches work items from a queue to available cores in a lock-free manner.
- IREE Runtime: A C-based library responsible for orchestrating operator execution and enabling parallelism within operators.
- Ariel OS: A Rust-based real-time operating system (RTOS) providing a lightweight, memory-safe environment for heterogeneous hardware.

The implementation relies on IREE v3.8.0. The scheduler limits the number of work items to avoid context-switching overhead and sets a minimum work item size to reduce inter-processor communication costs.

Key Contributions

First Rust Multi-Core TinyML Toolkit: Ariel-ML is presented as the first toolkit to natively support multi-core capabilities on microcontrollers for TinyML inference using a Rust runtime.
Open Source Implementation: The authors released the full source code, enabling cross-compilation, automated multi-core optimization, and deployment on 32-bit architectures (Arm Cortex-M, RISC-V).
Comprehensive Benchmarking: The paper provides reproducible benchmarks comparing Ariel-ML against prior art (RIOT-ML and a custom RIOT+IREE variant) across diverse hardware (Nordic nRF52840, ESP32-C3, Raspberry Pi Pico 1 & 2).

Experimental Results
The authors evaluated the system using quantized LeNet-5 and MCUNet models on single-core and dual-core MCUs.

Inference Latency: Ariel-ML demonstrated significant speedups on multi-core hardware. On dual-core RP2040 and RP2350 boards, it achieved up to a 1.6× speedup compared to single-core execution, approaching the theoretical limits imposed by bus contention. Compared to RIOT-ML on the same hardware, inference speeds improved by over 200%.
Memory Footprint: The integration of IREE and the Rust runtime introduced overhead. Flash usage increased by approximately 1.1–2.5×, and RAM usage increased by at least 1.2× compared to C-based RIOT-ML. However, the authors note that Ariel-ML achieves comparable memory footprints to pre-existing C/C++ toolkits when accounting for the advanced optimizations provided by IREE, which reduce the model artifact size itself.
Correctness: Numerical correctness was validated against PyTorch and TensorFlow references, with all deviations remaining below $10^{-8}$ .

Significance and Claims
The paper claims that Ariel-ML provides a necessary foundation for the future of TinyML on modern hardware. By automating parallelization on multi-core MCUs using a memory-safe language, it enables practitioners to:

Bridge the Hardware-Software Gap: Effectively utilize the increasing core counts in new MCU generations without manual, error-prone scheduling.
Enhance Security: Offer a safer alternative to C/C++ for critical systems software, aligning with emerging regulations like the EU Cyber Resilience Act.
Facilitate Deployment: Provide a reusable basis for researchers and developers to fine-tune IoT configurations and deploy neural networks directly on sensing/actuating systems.

The authors modestly acknowledge that while the current implementation shows promise, the memory overhead of the IREE runtime and Rust toolchain remains substantial. They identify future work in optimizing the runtime for smaller footprints and extending the platform to support on-device training and secure Over-the-Air (OTA) updates.