TinyContainer: Container Runtime Middleware Enabling Multi-tenant Microcontrollers with Built-in Security

This paper introduces TinyContainer, a lightweight middleware for multi-tenant microcontrollers that enables dynamic, fine-grained resource scheduling and security through a metadata-driven approach, demonstrating its effectiveness with a 4ms overhead in IoT environments and a specific TinyML use case.

The Gist

Imagine a tiny microcontroller chip (the brain of a smart thermostat, a fitness tracker, or a smart lock) as a small, crowded apartment.

In the past, this apartment could only have one tenant living there. If you wanted to add a new app or feature, you had to evict the old one and repaint the whole place (update the firmware). This was inefficient and rigid.

Recently, engineers figured out how to fit multiple "mini-apps" (containers) into this tiny apartment at the same time. This is called multi-tenancy. However, the current ways of doing this have two big problems:

1. The Landlord is too strict: Once the apartment is set up, you can't easily change who gets to use the kitchen (the device's sensors) or how long they can stay in the living room (execution time). To change these rules, you often have to evict everyone and rebuild the apartment.
2. No security guard: If one tenant gets crazy and tries to hog the TV or break the windows, the other tenants have no protection.

Enter TinyContainer.

The authors of this paper built a smart, lightweight property manager called TinyContainer specifically for these tiny microcontroller apartments. Here is how it works, using simple analogies:

1. The "Rulebook" (Metadata)

Instead of hard-coding rules into the apartment walls, TinyContainer gives every tenant a digital ID card and a rulebook (called metadata) when they move in.

• What it says: "Tenant A is allowed to use the front door (a specific sensor) but not the back door. Tenant B can use the back door but only for 5 seconds at a time."
• Why it's cool: The landlord (the device) can check this rulebook instantly. If a tenant tries to break the rules, the system stops them immediately without needing to rebuild the whole apartment.

2. The "Doorman" (Endpoints)

In the old days, if an app wanted to talk to a sensor (like a temperature gauge), it had to know exactly where the sensor was and how to open it. This was messy and unsafe.
TinyContainer introduces a Doorman system called Endpoints.

• How it works: The app doesn't talk to the sensor directly. It asks the Doorman, "Can I open the front door?"
• The Check: The Doorman checks the tenant's rulebook. If they have permission, the Doorman opens the door for them. If not, the Doorman says, "Nope, not allowed."
• The Result: This keeps the apps isolated. A malicious app can't sneak into a part of the house it wasn't invited to.

3. The "Timekeeper" (Scheduling)

Imagine a shared living room where everyone wants to watch TV. If one person (a "malicious" app) grabs the remote and refuses to let go, everyone else is stuck.
TinyContainer acts as a strict Timekeeper.

• It gives each tenant a specific amount of time to use the TV (the CPU).
• If a tenant tries to stay longer than their time limit, the Timekeeper hits a stopwatch (watchdog) and pulls the plug on that specific tenant, kicking them out of the living room so the others can watch.
• This ensures that even if one app crashes or tries to be greedy, the whole device doesn't freeze.

4. The "Specialist Service" (TinyML)

One of the paper's cool examples is Tiny Machine Learning (TinyML). This is when the device tries to "think" or recognize patterns (like identifying a specific sound or gesture).

• The Problem: Doing this thinking inside the tiny app (the container) is slow and heavy, like asking a child to do advanced calculus in their head.
• The TinyContainer Solution: The app keeps the "question" (the data) and the "answer key" (the model weights), but it sends the actual "thinking" to the Landlord's office (the native host system).
• The Analogy: The tenant writes down the math problem and the numbers they want to use, hands it to the Landlord, and the Landlord (who has a super-fast calculator) does the heavy lifting and hands the answer back.
• The Benefit: The app gets the answer quickly without needing to carry a heavy calculator, and the Landlord can handle the math much faster than the app could on its own.

The Bottom Line

The paper tested this system on real, tiny devices (like the ones found in Arduino boards). They found that:

• It's safe: It successfully stops bad apps from breaking things.
• It's flexible: You can change who gets access to what without reinstalling the whole device.
• It's fast enough: While adding these security checks takes a tiny bit of extra time (about 4 milliseconds per request), it is a small price to pay for keeping the device secure and running smoothly.

In short, TinyContainer turns a chaotic, one-size-fits-all microcontroller into a well-managed, secure apartment complex where multiple apps can live together without fighting, stealing resources, or crashing the building.

Technical Summary

Technical Summary: TinyContainer

Problem Statement

Software containerization on resource-constrained microcontrollers enables multi-tenant scenarios, allowing multiple applications with varying permission levels to share a single device. However, existing solutions face significant limitations in dynamic and heterogeneous environments:

1. Lack of Runtime Configuration: Current runtimes (e.g., Aerogel, CS4WAMR) typically rely on the underlying OS scheduler, preventing custom, per-container scheduling configurations.
2. Coarse-Grained Access Control: Access to host resources (native functions, peripherals) is often static. Deploying containers with different trust levels currently requires modifying firmware and reflashing the device, as permissions cannot be dynamically adjusted per container.
3. Inadequate Isolation: While memory isolation is common in WebAssembly (Wasm), temporal isolation and fine-grained control over host resource access remain difficult to implement on constrained devices.

Methodology

The authors propose TinyContainer, a lightweight container management middleware designed for multi-tenant microcontrollers. The system is built upon the following architectural and methodological pillars:

• Runtime Abstraction: TinyContainer utilizes an adaptor design pattern to decouple the management framework from specific runtimes. While it supports multiple runtimes (e.g., rBPF, JerryScript), the paper focuses on its integration with CS4WAMR, a Wasm runtime based on WAMR that offers memory segmentation.
• Metadata-Driven Management: Container behavior is governed by a metadata manifest encoded in CBOR (Concise Binary Object Representation) and signed using COSE (CBOR Object Signing and Encryption). This metadata defines:
  • Container Lifecycle: Start, loop, and stop entry points.
  • Scheduling Constraints: Execution time limits (start_exec_limit, loop_exec_limit) and call frequencies (loop_call_period, loop_max_calls).
  • Access Control: A syscall_mask defining authorized endpoint interactions.
  • Integrity: Cryptographic tokens (CWT) for code, data, and metadata to ensure authenticity.
• Endpoint System: To control access to host resources, TinyContainer implements an endpoint mechanism analogous to Unix I/O system calls (open, read, write, close). Containers request file descriptors for specific endpoints (peripherals or host services). The middleware validates these requests against the container's metadata permissions before routing them to the appropriate driver.
• Semi-Cooperative Scheduling: The framework employs a semi-cooperative scheduling model enforced by a watchdog timer. If a container exceeds its allocated execution time (defined in metadata), the watchdog triggers, and the container is terminated.

Key Contributions

1. TinyContainer Framework: A novel middleware providing container lifecycle management, per-container configurable scheduling, and fine-grained host resource access control via a metadata-driven approach.
2. Implementation on RIOT OS: The authors implemented TinyContainer on top of the RIOT OS, specifically integrating it with the CS4WAMR runtime to host small WebAssembly modules. The implementation is open-source.
3. Endpoint and Permission Architecture: A system allowing containers to access host services (including peripherals and native functions) through a controlled interface, enabling the deployment of co-located containers with different trust levels without firmware modification.
4. TinyML Use Case (TinyMLaaS): The paper demonstrates a "Tiny Machine Learning as a Service" architecture where model weights reside in the container, but inference is delegated to a native host driver. This leverages host-side optimizations without requiring Ahead-of-Time (AoT) compilation of the container or a full ML runtime (like TensorFlow Lite) on the device.
5. Empirical Evaluation: Benchmarks were conducted on popular IoT boards (Arduino Nano 33 BLE with nRF52840 and nRF9160-DK with nRF9160) featuring Cortex-M4 and Cortex-M33 microcontrollers.

Results

• Overhead:
  • Loading: Loading a container with TinyContainer incurs significant overhead due to metadata parsing and cryptographic verification (CWT validation). On the nRF9160-DK, loading metadata took ~88 seconds (without hardware crypto acceleration) and ~19 seconds (with software crypto), compared to ~1.6 seconds for raw WAMR.
  • Execution: The overhead for container execution is primarily driven by the endpoint routing mechanism. A single read() call to an endpoint took 4 ms with TinyContainer versus 11 µs with CS4WAMR directly.
  • Memory: The TinyContainer library adds approximately 8.8 KB of ROM and 16.9 KB of RAM for the base system. Each additional container adds roughly 156 bytes of ROM and 9.9 KB of RAM (mostly due to CS4WAMR buffer requirements).
• Scheduling Guarantees: Experiments with a "malicious" container (one that monopolizes execution) showed that while WAMR allows the container to block others, and CS4WAMR preempts it (leaving residual cost), TinyContainer successfully terminates the non-compliant container after the first timeout, allowing friendly containers to execute.
• TinyML Performance: Delegating inference to the host via the TinyContainer endpoint reduced inference time significantly compared to in-container interpretation. On the Arduino Nano 33 BLE, the TinyML endpoint achieved 512 ms inference time, compared to 48,324 ms for in-container WAMR interpretation. This approach avoids the need for AoT compilation while maintaining performance close to native execution.

Significance and Claims

The paper claims that TinyContainer fills a critical gap in the ecosystem of constrained multi-tenant microcontrollers by providing a framework for dynamic, fine-grained control over scheduling and resource access.

• Security: It enables the coexistence of mutually distrusting containers by enforcing strict temporal isolation and access control without requiring hardware-specific features like TrustZone.
• Flexibility: By abstracting the runtime and using metadata, it supports diverse container behaviors and allows for remote configuration of permissions and scheduling policies.
• Efficiency: The TinyML use case demonstrates that offloading compute-intensive tasks to the host via a controlled endpoint can mitigate the high overhead of Wasm interpretation without the complexity and trust requirements of AoT compilation.

The authors acknowledge limitations, including the high overhead of cryptographic verification on devices without hardware accelerators and the current requirement for containers to actively poll endpoints rather than being woken by them. However, they position TinyContainer as a foundational step toward secure, modular, and manageable multi-tenant IoT systems.