Orchestrating Multimodal DNN Workloads in Wireless Neural Processing

This paper proposes O-WiN, a framework for orchestrating multimodal DNN workloads in wireless neural processing that integrates wireless transmission and accelerator execution, demonstrating through the PACS algorithm that interleaving communication and computation significantly reduces end-to-end inference latency compared to sequential scheduling.

The Gist

Imagine you are running a high-end restaurant kitchen (the Edge Server) that needs to prepare a complex, multi-course meal (the Multimodal DNN) for a group of customers.

The ingredients for this meal don't come from your own pantry; they are being delivered by a fleet of delivery drivers (the Wireless Network) from different farms (the Sensor Nodes). Some drivers are bringing fresh vegetables (Text), others are bringing meat (Audio), and some are bringing exotic spices (Video).

The Problem: The "Wait-All" Kitchen

In the old way of doing things (which the paper calls RTFS), the head chef (the Accelerator) has a strict rule: "I cannot start chopping, cooking, or plating anything until every single ingredient has arrived at the kitchen door."

Even if the vegetables arrive 10 minutes early, the chef sits idle, staring at the empty counter, waiting for the last delivery truck to show up. Meanwhile, the delivery drivers are still stuck in traffic (wireless latency). This creates a huge gap where the kitchen is doing nothing, wasting time and energy.

The Solution: The "Pipelined" Kitchen

The paper proposes a new way of running the kitchen called O-WiN (Orchestrating Wireless Neural Processing). Instead of waiting for everything, the kitchen adopts a Pipelined approach (called PACS).

Here is how it works:

1. As soon as the vegetables arrive, the chef immediately starts chopping them.
2. While the vegetables are being chopped, the delivery drivers for the meat are still on the road.
3. The moment the meat arrives, the chef starts searing it, without waiting for the spices.
4. The spices arrive last, and the chef adds them to the dish just in time.

By overlapping the delivery (communication) with the cooking (computation), the kitchen stays busy the whole time. The "waiting" time is hidden because the chef is working on the ingredients that have already arrived.

The Two Strategies Compared

The paper tests two specific managers to see who runs the kitchen better:

1. The "Wait-All" Manager (RTFS)

• Strategy: "Don't start cooking until the last truck pulls in."
• Result: If one delivery is slow, the whole kitchen stops. The chef sits idle. This is simple but very slow when traffic is bad or deliveries are uneven.

2. The "Pipelined" Manager (PACS)

• Strategy: "Start cooking the moment any ingredient arrives. Keep the chefs busy."
• Result: This manager is smart. They know which ingredients are critical for the final dish. If the "Spices" (which are needed for the final step) are stuck in traffic, they prioritize getting those delivered first. If the "Vegetables" are easy to get, they let them come later.
• The Magic: This manager uses a "crystal ball" (a lightweight predictor) to guess which delivery will take the longest and focuses the delivery drivers on that one first, ensuring the chef never runs out of work.

Why This Matters in the Real World

In the world of AI, we often have to process different types of data at once (like a self-driving car processing camera images, radar signals, and GPS data all at once).

• The "Wireless Wall": Sometimes the internet connection is slow, and the data takes a long time to get to the computer.
• The Old Way: The computer waits for the data, then starts thinking. It's like a car waiting at a red light while the engine is off.
• The New Way (PACS): The computer starts thinking about the data it already has while the rest is still traveling over the internet. It's like driving the car while the traffic light is still red, but only if you have enough fuel (data) to move forward.

The Bottom Line

The paper shows that by coordinating the delivery trucks (wireless network) and the chefs (computer processors) to work simultaneously rather than sequentially, we can make AI much faster.

• When everything is balanced: The new way is slightly better.
• When things are messy (some data is huge, some is small, some connections are slow): The new way is dramatically faster. It hides the "waiting time" by keeping the computer busy with whatever data has arrived, effectively "hiding" the slow internet connection behind the speed of the computer.

In short: Don't wait for the whole pizza to arrive before you start eating the first slice. Start eating as soon as the first slice is delivered, and keep eating while the rest of the pizza is being baked and shipped. That is the power of PACS.

Technical Summary

1. Problem Statement

The paper addresses the inefficiency in Edge AI inference where wireless resource allocation and accelerator-level Deep Neural Network (DNN) scheduling are typically optimized in isolation.

• The Bottleneck: In multimodal scenarios (e.g., processing text, audio, and video simultaneously), end-to-end latency is jointly determined by wireless transmission time and DNN execution time. Current approaches often treat communication as a separate I/O step, leading to a "wait-all" barrier where the accelerator sits idle until all data from all sensor nodes arrives.
• The Challenge: There is a lack of end-to-end co-optimization between the wireless channel (OFDMA resource block allocation) and the multi-core accelerator (operator scheduling and core assignment). This prevents the effective overlapping of communication and computation, resulting in suboptimal latency, especially under high modality heterogeneity (where different data types have vastly different sizes and processing requirements).

2. Methodology

The authors propose Wireless Neural Processing (WNP), a paradigm that integrates wireless transmission and multi-core accelerator execution into a unified pipeline.

A. System Model

• Architecture: A system where sensor nodes transmit multimodal data via OFDMA to an edge server. The server hosts a multi-core accelerator (with a Network-on-Chip/NoC) executing a multimodal DNN modeled as a Directed Acyclic Graph (DAG).
• Key Constraints:
  • Wireless: Time-varying channels, limited spectrum (Resource Blocks), and dynamic uplink rates.
  • Computation: Precedence constraints (DAG), non-preemptive job execution, and shared NoC bandwidth among cores.
• Objective: Minimize the end-to-end inference makespan ($T_{makespan}$) by jointly optimizing Resource Block (RB) allocation and DNN job scheduling.

B. The O-WiN Framework

The authors introduce O-WiN, a modular orchestration framework with two tightly coupled stages:

1. Simulation-Based Optimization: An iterative loop where an optimization algorithm generates candidate policies (RB allocation + job mapping), and a performance evaluator simulates the full stack to estimate the makespan.
2. Runtime Execution: The optimized policy is deployed to manage real-time slice arrivals and core scheduling.

C. Proposed Algorithms

Two heuristic algorithms are developed to solve the NP-hard optimization problem:

1. RTFS (Release-Time First Scheduling):

   • Strategy: A sequential approach. It prioritizes minimizing the arrival time of the last data slice (tail latency) before starting any computation.
   • Mechanism: Uses a "wait-all" barrier. Computation only begins after all modalities are fully received. It uses a greedy RB allocation to minimize the maximum residual transmission time.
   • Limitation: Leaves accelerator cores idle during the transmission of the final, slowest modality.

2. PACS (Pipeline-Aware Co-Scheduling):

   • Strategy: A pipelined approach that interleaves communication and computation.
   • Mechanism:
     • Slice-Gated Execution: As soon as a specific modality's data slice arrives, its corresponding subgraph is released for execution, even if other modalities are still being transmitted.
     • Lightweight Predictor: Uses a max-plus dynamic programming predictor to estimate the impact of RB allocation on the final output completion time without running a full simulation.
     • Greedy Allocation: Assigns RBs to the slice that yields the largest marginal reduction in the predicted makespan, prioritizing data critical to synchronization points (e.g., fusion layers).

3. Key Contributions

1. Unified Modeling: Established a system architecture and mathematical model for Wireless Neural Processing (WNP), treating the wireless channel as a "remote memory" integrated into the accelerator's dataflow.
2. O-WiN Framework: Developed a scalable, modular framework that decouples communication and computation modules while enabling systematic cross-module coordination via simulation-based optimization.
3. Algorithmic Innovation:
   • Proposed RTFS as a baseline for sequential execution.
   • Proposed PACS to enable pipeline parallelism, effectively hiding wireless latency behind computation by overlapping data transfer with operator execution.
4. Comprehensive Evaluation: Conducted extensive simulations varying core counts, subcarrier numbers, and data heterogeneity to validate the approach.

4. Results

Simulation results demonstrate that PACS significantly outperforms RTFS, particularly in heterogeneous environments:

• Latency Reduction: PACS reduces end-to-end makespan by up to 19.1% compared to RTFS in scenarios with high modality heterogeneity (e.g., large differences in token sizes and compression ratios).
• Core Utilization: PACS eliminates the "idle gaps" seen in RTFS. By allowing early execution of available branches, it sustains higher NoC bandwidth utilization and keeps cores busy while waiting for remaining data.
• Sensitivity Analysis:
  • The performance gain of PACS is most pronounced when the number of OFDMA subcarriers (communication resources) varies, confirming that the system is often communication-constrained.
  • PACS excels when modalities have imbalanced loads (e.g., one modality is large/slow to transmit but fast to compute, or vice versa), as it can mask the slow transmission of one modality by computing others in parallel.
• Homogeneous Cases: When modalities are balanced, the gain is minimal, and in rare cases, the overhead of PACS's prediction might slightly outweigh the benefits, though it generally remains competitive.

5. Significance

This paper represents a fundamental shift in Edge AI architecture design:

• Breaking the "Wireless Wall": It moves beyond treating wireless transmission as a simple pre-processing step, instead integrating it into the critical path of DNN execution.
• Pipeline Parallelism: It demonstrates that communication-computation pipelining is a viable and highly effective strategy for reducing latency in multimodal edge inference, challenging the traditional "compute-as-a-service" abstraction.
• Practical Impact: The proposed framework and algorithms provide a blueprint for next-generation edge servers to handle diverse, real-time sensor data streams with lower latency and higher energy efficiency, crucial for applications like autonomous driving, industrial IoT, and augmented reality.