🤖 machine learning

Managing Task Execution for Unknown Workloads in Batteryless IoT: A Hardware-Agnostic Evaluation

This paper proposes and evaluates two hardware-agnostic, dynamic scheduling strategies—a model-free Reinforcement Learning agent and an on-the-fly Approximated Prediction method—that effectively manage task execution for batteryless IoT devices with unknown workloads, revealing distinct trade-offs in throughput, resilience, and computational cost compared to existing adaptive and static approaches.

Original authors: Samer Nasser, Henrique Duarte Moura, Ritesh Kumar Singh, Maarten Weyn, Jeroen Famaey

Published 2026-06-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Samer Nasser, Henrique Duarte Moura, Ritesh Kumar Singh, Maarten Weyn, Jeroen Famaey

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a tiny, battery-less robot that runs entirely on solar power. It doesn't have a big battery to store energy for a rainy day; instead, it has a small "energy bucket" (a supercapacitor) that fills up when the sun shines and empties when the robot works.

The problem is that the sun is unpredictable. Sometimes it's bright, sometimes it's cloudy, and sometimes it's night. The robot also has to do tasks, like sending a message (a "LoRa transmission"), but it doesn't know exactly how much energy that task will cost. If it tries to send a message when its bucket is too empty, the bucket runs dry, the robot shuts down, and it has to wait hours to recharge before it can wake up again.

This paper is about teaching the robot how to decide when to work and when to sleep without knowing the future or having a manual for its specific hardware.

The Problem: The "Black Box" Dilemma

Usually, engineers program robots with a fixed rule: "If the energy bucket is above 50%, send a message. If it's below, sleep."
But this is like trying to drive a car with a fixed speed limit regardless of the road conditions.

Too strict: You might sleep when you could have sent a message.
Too loose: You might try to send a message when you don't have enough power, causing a crash (shutdown).
The "Black Box": The robot doesn't know how much energy a message will cost because the distance to the receiver changes, or the message size changes. It's a mystery.

The Solutions: Three New Ways to Decide

The researchers tested four different "brains" to manage this robot. They didn't just want to prove one was the best; they wanted to see the trade-offs of each.

1. The "Gambler" (Approximated Prediction - AP)

This method is like a smart gambler who looks at the last few bets to guess the next one.

How it works: After the robot finishes a task, it measures exactly how much energy it used. It uses that number to guess how much the next task will cost. It then checks its bucket: "If I do this next task, will I still have enough left to stay awake?"
The Result: It's incredibly fast and efficient. It gets almost as many tasks done as a "Magic Oracle" (a perfect predictor that knows the future), but it doesn't need any prior knowledge. It's lightweight and works great on small, tight budgets.

2. The "Student" (Reinforcement Learning - RL)

This is a robot that learns by trial and error, like a child learning to ride a bike.

How it works: The robot tries different strategies. If it crashes (runs out of power), it gets a "punishment." If it stays awake and sends messages, it gets a "reward." Over time, it learns a complex rulebook on how to balance working and sleeping.
The Result: It's very flexible. You can teach it to be aggressive (send as many messages as possible) or conservative (never shut down). However, it's heavy; it requires a lot of computer power and memory to run, which uses up some of the robot's own energy.

3. The "Pacer" (AsTAR)

This method is like a marathon runner who knows they have to run for 24 hours straight.

How it works: Instead of looking at the current energy level, it looks at the trend. If the energy is dropping, it slows down. If it's rising, it speeds up. It uses a specific math trick (Additive Increase, Multiplicative Decrease) to keep the energy level steady.
The Result: It is the best at "pacing." It is the only method that successfully keeps the robot working through the night without shutting down, bridging the long gap when the sun isn't shining.

4. The "Old School" (Static Threshold)

This is the traditional method: "If the bucket is above X, work. If below, sleep."

The Result: It works surprisingly well if you have a huge bucket (a large capacitor). If you have a lot of extra energy, you don't need a smart brain; a simple rule works fine. But if your bucket is small, this method fails miserably, either shutting down too often or missing opportunities.

The Big Takeaways

The paper concludes that there is no single "best" brain. The right choice depends entirely on the size of your energy bucket:

If you have a small bucket (severe constraints): You need the smart methods. The Gambler (AP) is the best choice because it's fast, smart, and doesn't waste energy on complex calculations. The Student (RL) is good if you need to fine-tune exactly how risky you want to be.
If you have a big bucket (plenty of room): You don't need a supercomputer. A simple Old School rule is actually better because it saves energy and memory. The fancy methods are overkill here.
If you need to work all night: The Pacer (AsTAR) is the only one that can reliably keep the robot running through long periods of darkness.

The Bottom Line

The researchers built a realistic simulator using real solar data from Iceland and tested these methods on different sizes of "energy buckets." They found that while high-tech AI and prediction methods are amazing for tiny, constrained devices, sometimes the simplest solution is the most efficient if you have enough resources to spare. The key is matching the complexity of the brain to the size of the energy tank.

Technical Summary: Managing Task Execution for Unknown Workloads in Batteryless IoT

1. Problem Statement

The rapid proliferation of Internet of Things (IoT) devices has raised significant environmental and economic concerns regarding the use of Li-ion batteries, which contain toxic materials and require frequent replacement. While batteryless, energy-harvesting architectures offer a sustainable alternative by utilizing supercapacitors, they introduce a critical challenge: managing highly volatile energy storage.

Unlike batteries, supercapacitors exhibit non-linear voltage drops proportional to energy expenditure. Coupled with unpredictable ambient energy harvesting (e.g., solar) and "black box" applications with dynamic, unknown energy profiles (e.g., LoRa transmissions with variable payloads and spreading factors), traditional static scheduling strategies fail. Static methods rely on fixed voltage thresholds or pre-measured task profiles, which are either insufficient for dynamic conditions or require hardware-specific manual calibration. The core problem is to develop hardware-agnostic, dynamic scheduling strategies that can intelligently manage task execution without prior knowledge of energy consumption, ensuring system resilience (avoiding power failures) while maximizing task throughput.

2. Methodology

System Modeling

The authors model the batteryless IoT system as a Markov Decision Process (MDP). The system state is defined by the capacitor voltage ( $V(t)$ ), harvesting current ( $I_H$ ), and the decision to execute a task ( $E$ ) or sleep ($NE$).

Capacitor Dynamics: A closed-form RC step response equation is derived to predict voltage evolution ( $V(t+\Delta t)$ ) based on harvesting current, load resistance, and capacitance.
Failure Conditions: If the voltage drops below a minimum operational threshold ( $V_{min}$ ), the system enters an OFF-state (power failure), requiring a prolonged recovery period to recharge past the turn-on threshold ( $V_{to}$ ).

Proposed Strategies

The paper evaluates four distinct decision logic approaches:

Static Thresholding: A baseline approach where tasks execute only if $V(t) \geq V_{thresh}$ . This requires manual tuning and prior knowledge of task costs to be effective.
AsTAR (Adaptive Task Rate): An existing approach using Additive Increase Multiplicative Decrease (AIMD) and Multiplicative Increase Additive Decrease (MIAD) principles. It adjusts the task execution rate based on voltage deviations from a target, prioritizing long-term pacing over immediate throughput.
Reinforcement Learning (RL) Agent: A model-free approach using Proximal Policy Optimization (PPO).
- Observation Space: Includes capacitor voltage, harvesting current, task duration, time since failure, and capacitor size.
- Training: Employs domain randomization across capacitor sizes (0.5F to 10F) to ensure hardware-agnosticism.
- Reward Functions: Two variants are tested: one optimizing for Inter-Task Interval (ITI) (minimizing gaps between tasks) and another optimizing for Off-Time (minimizing power failure duration).
Approximated Prediction (AP): A novel, lightweight method that treats the application as a "black box."
- It approximates the equivalent load resistance ( $R_{eq}$ ) of the current task by solving a root-finding problem (Newton-Raphson) using voltage measurements before and after the previous task.
- It predicts the future voltage if the next task is executed. If the prediction exceeds a padded safety threshold, the task runs; otherwise, the system sleeps.
- It includes a dynamic recalibration mechanism triggered by power failures or significant prediction errors.

Experimental Framework

The evaluation utilizes a physically accurate simulation framework driven by:

Real-world Data: Solar harvesting data from Iceland (2022–2023) converted to harvesting current.
Dynamic Workloads: LoRa transmission profiles with variable payloads and Adaptive Data Rate (ADR) mechanisms that change power consumption based on simulated channel conditions (RSSI).
Hardware Constraints: Simulated STM32L4 MCU and SX1262 LoRa module with specific energy overheads for the decision logic itself (e.g., RL inference time, AP solver iterations).
Variables: Capacitor sizes ranging from 0.5F to 10F.

3. Key Contributions

Hardware-Agnostic "Black Box" Scheduling: The proposal of two novel strategies (RL and AP) that manage energy budgets without prior knowledge of task energy consumption or specific capacitor sizes.
Comprehensive Simulation Framework: Development of an open-source, physically accurate simulator coupling non-linear capacitor dynamics with real-world solar data and variable LoRa loads.
Operational Trade-off Analysis: A detailed comparative analysis exposing the distinct strengths and weaknesses of dynamic methods versus static baselines, specifically regarding throughput, survival time, and execution pacing.
Identification of Practical Limits: The demonstration that advanced dynamic strategies are most valuable under severe hardware constraints (small capacitors), while larger energy buffers can often rely on simpler static policies.

4. Results and Findings

Throughput vs. Reliability ("Boom and Bust")

High Throughput: The Approximated Prediction (AP) approach and the Short-Term Oracle achieved the highest number of daily successful executions (exceeding 1000 tasks/day for capacitors >1.5F). Notably, AP achieved near-Oracle throughput without prior knowledge.
Reliability Cost: These high-throughput methods exhibited significantly lower median times between power failures ("off-states") compared to more conservative methods. They maximize data transmission but risk frequent system blackouts.

Execution Pacing

AsTAR Superiority: The AsTAR approach uniquely excelled at "pacing." While it had a lower average execution rate, it was the only method that successfully bridged long energy gaps (e.g., nighttime) without communication blackouts. For capacitors >2.5F, AsTAR reduced the maximum daily inter-task interval to ~1 hour, whereas other methods flatlined at ~8 hours (the duration of night).

System Resilience and Recovery

Survival: The RL Agent (Off-Time optimized) and Static (3.45V) approaches effectively eliminated power failures for most capacitor sizes, maximizing total device uptime.
Recovery Speed: In severely constrained scenarios (0.5F–1F), the AP and RL agents demonstrated faster recovery from power failures compared to static baselines, thanks to their dynamic adaptation.

Hardware Agnosticism

Static Limitations: Static thresholds failed to generalize; a low threshold (1.9V) caused frequent failures in small capacitors, while a high threshold (3.45V) wasted energy in larger capacitors.
Dynamic Success: Both RL and AP maintained a balanced trade-off between survival and throughput across the entire 0.5F–10F range without manual recalibration.
Overhead: The AP approach was highlighted as particularly efficient, offering hardware-agnosticism with minimal computational and memory overhead compared to the RL agent's neural network inference requirements.

5. Significance and Claims

The paper modestly claims that no single decision logic is universally superior. Instead, the significance lies in identifying the specific operational contexts where each method provides the most value:

For Severely Constrained Systems (Small Capacitors): Advanced dynamic strategies are essential. The AP method is recommended for its lightweight, real-time adaptation that matches optimal throughput without calibration. The RL approach is suitable when a tunable balance between execution and survival is required.
For Night-Time Pacing: AsTAR remains the preferred strategy for applications requiring consistent communication across long harvesting gaps.
For Systems with Large Energy Buffers: The paper concludes that for devices with larger capacitors (e.g., >5.5F in their scenario), simple static thresholding is highly effective. In these less constrained systems, the computational and memory overhead of advanced dynamic methods does not justify the marginal benefits over optimized static policies.

The work establishes a boundary for the practical utility of dynamic thresholding, suggesting that the complexity of AI-driven or predictive scheduling should be reserved for scenarios where hardware constraints (small energy buffers) make static optimization impossible.