💻 computer science

ECORE: Energy-Conscious Optimized Routing for Deep Learning Models at the Edge

The paper proposes ECORE, an energy-conscious routing framework for edge computing that dynamically directs image processing tasks to optimal device-model pairs using novel estimation and greedy algorithms, achieving significant reductions in energy consumption and latency with minimal accuracy loss.

Original authors: Daghash K. Alqahtani, Maria A. Rodriguez, Muhammad Aamir Cheema, Hamid Rezatofighi, Adel N. Toosi

Published 2026-02-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Daghash K. Alqahtani, Maria A. Rodriguez, Muhammad Aamir Cheema, Hamid Rezatofighi, Adel N. Toosi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are running a busy security checkpoint at a large airport. You have a fleet of different security scanners (some are super-fast but basic, others are slow but incredibly detailed) and a team of guards with different levels of training.

Every time a person walks up to the line, they carry a bag. Sometimes the bag is empty; sometimes it's packed with 50 items.

The Problem:
If you use your most powerful, high-tech scanner on an empty bag, you waste a massive amount of electricity and time. But if you use a basic scanner on a bag full of dangerous items, you might miss something important. The challenge is figuring out, instantly, which scanner and which guard is the perfect match for that specific bag without wasting energy or missing a threat.

The Solution: ECORE
The paper introduces a system called ECORE (Energy-Conscious Optimized Routing). Think of ECORE as a super-smart traffic controller standing at the front of the line. Its job is to look at every incoming bag (image) and instantly decide: "Is this a simple bag or a complex one?" and then send it to the right scanner.

Here is how it works, using simple analogies:

1. The "Quick Glance" (Estimation)

Before the bag goes to a scanner, the traffic controller (ECORE) does a very quick, low-energy "glance" at it to guess how many items are inside. It doesn't open the bag or scan it deeply; it just looks for shapes or edges.

Analogy: Imagine a guard who just looks at the outline of the bag. If it looks flat and thin, they guess it's empty. If it looks bulky and lumpy, they guess it's full.
The Paper's Methods: The researchers tried three ways to do this "glance":
- Edge Detection: Looking for outlines (like tracing the edge of a shadow).
- Lightweight Scanner: Using a tiny, fast version of a scanner just to count items.
- The "Memory" Trick: If the last bag had 3 items, and this one looks similar, they assume it also has 3 items. This saves energy because they don't have to "glance" at every single bag.

2. The "Smart Routing" (The Decision)

Once the controller guesses the number of items, it consults a "menu" of options. This menu lists every possible combination of scanner and guard, along with how much electricity they use and how accurate they are.

The Rule: If the bag looks simple (few items), the controller sends it to the energy-saving scanner. If the bag looks complex (many items), it sends it to the high-accuracy scanner.
The Safety Net: The system has a "tolerance setting." It says, "It's okay if the accuracy drops by just a tiny bit (like 2%) if it saves us a lot of electricity." This ensures we never miss a critical threat, but we don't waste power on easy tasks.

3. The Results (The Race)

The researchers tested this system in a real-world lab using various devices (like Raspberry Pis and Jetson computers) and thousands of real images (like people crossing streets or crowds in a city).

They compared their "Smart Controller" against other methods:

The "Random" Method: Sending bags to scanners randomly.
The "Always Best" Method: Using the most powerful scanner for every bag, regardless of need.
The "Always Cheapest" Method: Using the weakest scanner for every bag to save power.

The Outcome:
ECORE won the race. By being smart about when to use the heavy machinery, it achieved:

35% less energy used (saving battery life).
49% faster processing (less waiting time).
Only a 2% drop in accuracy (it was still almost as good as the "Always Best" method).

Why This Matters

In the real world, many devices (like security cameras in smart cities or sensors in farms) run on batteries or solar power. They can't afford to run their most powerful engines 24/7. ECORE acts like a smart thermostat for AI: it turns the power up only when the room gets cold (complex scene) and turns it down when the room is warm (simple scene).

In short: ECORE is a system that stops AI from overworking itself. It looks at a picture, guesses how hard the job will be, and sends it to the most efficient worker available, saving huge amounts of energy without missing the important details.

Technical Summary: ECORE – Energy-Conscious Optimized Routing for Deep Learning Models at the Edge

Problem Statement

The deployment of deep learning-based object detection models (e.g., YOLO, SSD, EfficientDet) on resource-constrained edge devices presents a critical challenge: balancing energy consumption with detection accuracy. Real-world visual scenes vary significantly in complexity, from sparse environments to dense, crowded intersections. However, static deployment strategies often apply a single device-model pair regardless of scene complexity, leading to unnecessary energy expenditure in simple scenarios or potential accuracy degradation in complex ones.

The core problem addressed is how to minimize energy usage on edge devices without degrading detection quality, particularly in safety-critical applications like surveillance and traffic monitoring where accuracy is non-negotiable. Existing solutions often fail to leverage the heterogeneity of available edge resources (different devices and model variants) dynamically based on the specific content of incoming image requests.

Methodology

The authors propose ECORE (Energy-Conscious Optimized Routing for Deep Learning Models at the Edge), a framework that dynamically routes image processing requests to the most suitable device-model pair based on estimated scene complexity.

System Architecture

ECORE operates in a gateway-centric architecture where a central gateway receives image requests from distributed cameras or sensors. The gateway performs lightweight preprocessing to estimate the number of objects in an image (scene complexity) and then executes a routing algorithm to select the optimal backend edge device and model configuration. The system assumes a pool of heterogeneous edge devices (e.g., Raspberry Pi 4/5, Jetson Orin Nano, TPU accelerators) running various object detection models.

Object Count Estimation

To avoid negating energy savings with heavy preprocessing, ECORE employs three lightweight estimation techniques:

Edge Detection (ED): Uses the Canny edge detection algorithm to estimate object counts based on intensity changes, offering a computationally efficient but coarse approximation.
SSD-Based Front-End (SF): Deploys a lightweight SSD model directly at the gateway to count objects, providing higher accuracy at a moderate computational cost.
Output-Based (OB): Leverages temporal continuity in video streams by reusing the object count from the previous frame's detection results, assuming consecutive frames have similar object densities. This avoids per-frame estimation overhead.

Routing Algorithm

The routing logic is formulated as a constrained optimization problem. Given an estimated object count group ( $G$ ), the algorithm:

Identifies the subset of device-model pairs ( $M_G$ ) relevant to that group.
Determines the maximum achievable mean Average Precision (mAP) within this subset ( $mAP_{max}$ ).
Applies a dynamic tolerance margin ( $\delta_{mAP}$ ) to define a feasible set ( $F$ ) of pairs where $mAP_i \geq mAP_{max} - \delta_{mAP}$ .
Selects the pair within $F$ that minimizes energy consumption ( $e_i$ ).

This greedy approach ensures that the system trades a controlled, minimal reduction in accuracy for significant energy gains, provided the accuracy remains within the acceptable tolerance.

Key Contributions

ECORE Framework: A novel system architecture for energy-efficient routing of image processing tasks across heterogeneous edge devices, coordinated by a central gateway.
Estimation-Based Routing: The introduction of three distinct object count estimation techniques (ED, SF, OB) to enable context-aware routing without heavy computational overhead.
Comprehensive Evaluation: An extensive experimental evaluation on a real-world testbed using diverse datasets (COCO validation, a balanced sorted dataset, and a pedestrian video stream) and multiple baseline comparisons.

Experimental Results

The framework was evaluated using a testbed comprising eight device configurations (Raspberry Pi 3/4/5 with/without TPU/AI Hat, Jetson Orin Nano) and various models (SSD, EfficientDet, YOLOv8).

Energy and Latency Reduction: Compared to accuracy-centric methods (specifically the Highest mAP Per Group router), ECORE's routing strategies reduced energy consumption by 35% and latency by 49%.
Accuracy Trade-off: These improvements were achieved with a marginal loss in detection accuracy of only 2% compared to accuracy-centric baselines.
Method Performance:
- On static image datasets (COCO), the Edge Detection (ED) approach offered the best trade-off, achieving accuracy close to the Oracle router (which has perfect ground truth knowledge) while maintaining competitive energy efficiency.
- On sequential datasets (Balanced Sorted and Video), the Output-Based (OB) approach proved most effective, leveraging temporal continuity to minimize overhead and energy while maintaining high accuracy.
Tolerance Sensitivity: Setting the accuracy tolerance ( $\delta_{mAP}$ ) to 5% yielded significant reductions in energy and latency while keeping the actual accuracy drop around 2%.

Significance and Claims

The paper claims that ECORE demonstrates the feasibility of dynamic, context-aware routing as a viable strategy for enabling energy-efficient AI workloads at the edge. By acknowledging that not all scenes require the same computational power, the framework allows systems to adaptively select lightweight models for simple scenes and more robust models for complex ones.

The authors emphasize that this approach is particularly significant for edge devices operating under strict power constraints (e.g., battery-powered or solar-powered sensors). The study asserts that it is possible to minimize energy usage without degrading detection quality in a manner that compromises safety-critical applications, provided that the routing mechanism is lightweight and the accuracy tolerance is carefully managed. The work distinguishes itself from prior research by operating entirely at the edge, focusing on content-aware selection of device-model pairs rather than just system-level orchestration or single-device adaptation.