Technical Summary: Adaptive DNN Partitioning and Offloading in Heterogeneous Edge-Cloud Continuum

Problem Statement

The proliferation of Internet-of-Things (IoT) devices has generated massive data volumes requiring sophisticated processing via Deep Neural Networks (DNNs). While centralized cloud execution offers computational power, it introduces prohibitive latency, bandwidth constraints, and privacy risks for real-time IoT applications. Conversely, executing DNNs entirely on resource-constrained edge devices often leads to high energy consumption and slow inference times.

Existing solutions for distributing inference across the edge-cloud continuum typically rely on static partitioning strategies. These approaches define fixed split points for neural network layers prior to deployment, failing to adapt to runtime dynamics such as fluctuating network bandwidth, node mobility, or varying computational loads. Furthermore, a significant gap exists in the literature: most proposed frameworks are evaluated in simulated environments assuming ideal network conditions, with few validated on physical hardware. Additionally, many real-world studies focus primarily on latency or task completion time, often neglecting energy consumption as a primary optimization objective.

Methodology

The authors propose an adaptive partitioning and offloading framework designed to dynamically split DNN inference workloads across a heterogeneous three-tier continuum (Edge, Fog, and Cloud) to minimize energy consumption while adhering to strict latency constraints.

System Architecture

The framework operates on a physical testbed consisting of:

• Edge: Raspberry Pi 4 (resource-constrained).
• Fog: Laptop (Intel Core i7).
• Cloud: High-performance Desktop PC (NVIDIA RTX 4070 Ti GPU).

The architecture allows for partitioning schemas where $i$ layers run on the edge, layers $i+1$ to $j$ run on the fog, and layers $j+1$ to $N-1$ run on the cloud.

Algorithmic Pipeline

The framework follows a five-stage pipeline:

1. Offline Profiling: Before runtime, the system profiles the model to generate two lookup tables:

   • Activation Size Table: Records the size of intermediate tensors at every feature boundary to estimate communication costs.
   • Relative Compute Weight Table: Assigns normalized fractions of total inference work to each layer based on a single execution, serving as a stable reference for scaling runtime measurements.

2. Two-Point Link Probing: To estimate network conditions without heavy overhead, the system transmits two payloads of contrasting sizes ($s_1, s_2$) across network hops. By averaging round-trip times, it models the link as $rtt(s) = \omega + s/\beta$, deriving the fixed overhead ($\omega$) and effective throughput ($\beta$).

3. Candidate Split Estimation: For any candidate split $(i, j)$, the system predicts end-to-end latency and energy. Latency is the sum of computation times (derived from profiled weights and node speeds) and transfer times (derived from link models). Energy is estimated by multiplying node-specific compute times by empirical power rates.

4. Best Candidate Search: The system evaluates all valid split pairs. Candidates are filtered to ensure they meet the latency deadline ($L_{max}$) and do not perform worse than a static baseline. The optimal split minimizes a weighted objective score:
   $$S(i, j) = w_{end}\frac{\hat{E}_{end}}{n_{end}} + w_{tot}\frac{\hat{E}_{tot}}{n_{tot}} + w_{lat}\frac{\hat{L}}{n_{lat}}$$
   where weights prioritize edge energy consumption (0.6–0.9) over total system energy and latency.

5. Adaptive Distributed Inference Scheduling:

   • Initialization: The system runs a baseline split and three "probe" splits (edge-heavy, balanced, cloud-heavy) to fit per-node compute rates and link models.
   • Steady State: The system executes the current split for a window of inferences. Periodically, it re-evaluates the environment by re-probing links and re-fitting rates. If a new candidate improves the objective score by at least 3% (or if the latency deadline is violated), the system switches to the new partition. If no better candidate exists during a deadline violation, it falls back to the static baseline.

Key Contributions

The paper outlines three primary contributions:

1. Adaptive Framework: A novel framework for DNN inference that dynamically partitions workloads across a heterogeneous edge-cloud continuum, prioritizing energy efficiency without violating latency constraints.
2. Physical Testbed: The construction and utilization of a fully functional three-tier testbed (Raspberry Pi, Laptop, Desktop PC) to evaluate vertical offloading strategies on real hardware, addressing the reliance on simulations in prior work.
3. Empirical Evaluation: Comprehensive real-world experiments using three widely adopted CNNs (VGG16, AlexNet, MobileNetV2), collecting hardware-measured energy and latency metrics across single-device, static, and adaptive configurations.

Results

The framework was evaluated against two baselines: single-device execution and static partitioning (fixed equal workload distribution).

• Energy Reduction: The adaptive framework achieved significant energy savings compared to the static baseline:
  • VGG16: 35.82% reduction (from 5.69 J to 3.65 J).
  • AlexNet: 35.70% reduction (from 0.68 J to 0.43 J).
  • MobileNetV2: 27.09% reduction (from 0.92 J to 0.67 J).
• Latency Reduction: End-to-end latency was also reduced:
  • VGG16: 6.34% reduction.
  • AlexNet: 22.92% reduction.
  • MobileNetV2: 14.20% reduction.
• Observations: For MobileNetV2, static partitioning actually increased latency compared to single-device execution due to network overhead; the adaptive framework successfully mitigated this by shifting the split point. The results consistently show that energy reductions exceed latency reductions, reflecting the framework's objective weighting.

Significance and Claims

The paper claims that its findings confirm the superiority of adaptive partitioning over static methods in dynamic, heterogeneous environments. By validating the approach on real hardware rather than simulation, the work addresses a critical gap in the literature regarding the practical performance of offloading strategies.

The authors assert that their framework successfully minimizes energy consumption on resource-constrained edge devices without violating real-time latency constraints. Unlike Reinforcement Learning (RL) based approaches or complex parallel inference architectures, this framework relies on a lightweight, periodic evaluation of a single weighted objective function, making it continuously adaptive to environmental changes such as network bandwidth fluctuations and node load variations. The results are presented as comparable to state-of-the-art works in latency reduction but with a distinct focus on energy efficiency, a metric often overlooked in existing heterogeneous edge-cloud studies.