💻 computer science

AMORL: A Training-Aware, Adaptive, Multi-Objective RL-CNN Framework for Dynamic DNN Mapping on Networks-on-Chip

This paper introduces AMORL, a training-aware, adaptive framework combining reinforcement learning and convolutional neural networks to dynamically map diverse DNN workloads onto heterogeneous Networks-on-Chip, achieving significant reductions in communication cost, load imbalance, and latency while maintaining high throughput and energy efficiency.

Original authors: Md Farhadur Reza

Published 2026-08-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Md Farhadur Reza

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern world of computing, the most powerful machines are no longer defined by a single, massive brain, but by thousands of smaller processors working in unison. Imagine a city where millions of tiny workers must constantly pass notes to one another to solve a single, complex problem. If the roads between them are clogged, the entire city grinds to a halt. This is the reality of the deep neural networks that power today's artificial intelligence, from image recognition to language translation. These systems are built from layers of artificial neurons that communicate in vast, intricate patterns. To run them efficiently, engineers place these neurons onto specialized chips called many-core systems, where the processors are connected by a network of tiny wires known as a Network-on-Chip. This internal network acts as the chip's circulatory system, carrying data between processors. However, as these artificial brains grow larger and more complex, the traffic on these internal roads becomes a bottleneck. Traditional methods of deciding which processor handles which part of the calculation often fail to account for the unique, shifting demands of different AI models, leading to wasted energy, overheating, and slow performance.

A researcher at Eastern Illinois University has developed a new approach to solve this traffic problem, a system they call AMORL. Instead of using a fixed set of rules to assign tasks to processors, this system learns how to map the flow of data dynamically, much like a smart traffic control center that adjusts signal timings in real-time based on current congestion. The researcher trained their system using a combination of two powerful techniques: reinforcement learning, which allows a computer to learn through trial and error, and a convolutional neural network, which helps the system recognize spatial patterns in how data moves. By teaching the system to look at the entire chip as a map, it can identify which processors are idle and which are overloaded, then instantly reroute the flow of information to keep everything running smoothly.

The core innovation of AMORL lies in its ability to adapt to the specific needs of different artificial intelligence models and the changing conditions of the chip itself. The researcher tested their system on six distinct types of neural networks, ranging from simple models used for basic image classification to massive, complex architectures like ResNet-50 and BERT-tiny. They simulated these models running on chip networks of varying sizes, from small grids of 16 processors to massive arrays of over a thousand. The system was designed to handle the entire lifecycle of a training session, recognizing that the communication needs of an artificial brain change as it learns. During the initial phase of learning, the system sends data forward; later, it sends error signals backward to correct mistakes; and finally, it updates its internal weights. AMORL learns to optimize the placement of tasks for each of these phases, ensuring that the most critical data travels the shortest possible distance.

In their simulations, the results were striking. Compared to older mapping strategies that relied on random assignment or standard optimization algorithms, the AMORL framework reduced the cost of communication by more than 70 percent. This means the data had to travel significantly fewer hops between processors, drastically cutting down the time it took for the system to complete a task. The system also improved load balancing by up to 86 percent, ensuring that no single processor was left sitting idle while others were overwhelmed. Perhaps most importantly, the system managed to do this while keeping energy consumption low and preventing the chip from overheating. The researcher incorporated a thermal safety metric into their learning process, allowing the system to automatically shift tasks away from processors that were getting too hot, thus avoiding the need for the chip to slow down to cool off.

To verify that these findings held up under realistic conditions, the researcher ran their mappings through a highly detailed, cycle-accurate simulator that mimics the physical behavior of the chip's network. They tested the system on a 4x4 grid of processors and a larger 8x8 grid, pushing the injection rate of data to see how the system performed under heavy traffic. On the smaller grid, the system sustained a throughput of up to 876 gigabits per second. On the larger 8x8 grid, it achieved a staggering throughput of over 2.77 terabits per second. These numbers indicate that the system not only works in theory but maintains high efficiency even when the network is under immense pressure. The simulations confirmed that the mappings generated by AMORL could handle the diverse and unpredictable demands of real-world artificial intelligence workloads without requiring the system to be retrained from scratch every time a new model was introduced.

The researcher also explored how different components of their system contributed to its success. They found that the part of the system responsible for dynamically weighing different goals—such as speed versus energy efficiency—was the most critical factor. When they removed this adaptive weighting, the system's performance dropped significantly, proving that a one-size-fits-all approach is insufficient for these complex tasks. Similarly, the component that recognized spatial patterns on the chip was essential for reducing communication costs, especially as the chip size increased. The system also demonstrated a remarkable ability to adapt quickly to sudden changes in workload. If the type of data or the size of the task changed mid-operation, the system could make rapid adjustments without needing a full retraining cycle, ensuring continuous high performance.

This work represents a significant step forward in making artificial intelligence hardware more efficient and adaptable. By teaching the chip itself how to organize its own internal traffic, the researcher has created a framework that can scale from small, simple networks to the massive, complex systems of the future. The ability to jointly optimize for speed, energy, and heat management means that future AI accelerators could run faster and cooler, handling a wider variety of tasks without the need for constant human intervention. The findings suggest that as we move toward chips with thousands of cores, the key to unlocking their full potential lies not just in adding more power, but in teaching them how to share that power intelligently. The AMORL framework provides a blueprint for this intelligence, showing that with the right learning strategies, these complex systems can find their own optimal paths through the digital landscape.

Technical Summary: AMORL Framework for Dynamic DNN Mapping on NoCs

Problem Statement

As transistor technology advances, many-core systems integrating hundreds to thousands of cores are becoming standard for executing Deep Neural Networks (DNNs). These systems rely on Networks-on-Chip (NoCs) as the primary communication fabric. However, traditional mapping strategies struggle to handle the diverse and intensive communication patterns inherent in DNNs, such as global all-to-all exchanges in fully connected layers and localized patterns in convolutional layers.

Existing mapping approaches face three primary limitations:

Lack of Scalability: Many techniques are tailored to fixed NoC sizes or specific DNN architectures, lacking the flexibility to adapt to variable dimensions (e.g., from 4×4 to 32×32 meshes) or diverse model depths.
Static Optimization: Traditional methods often fail to address the multi-objective nature of the problem, which requires simultaneous optimization of conflicting goals: minimizing communication latency, balancing computational loads, reducing energy consumption, and managing thermal risks.
Inflexibility to Dynamic Workloads: Real-world accelerators encounter dynamic contexts, including rapid switches between different models, varying batch sizes, and changing training phases (forward, backward, weight update). Static mappers trained on a single dataset (e.g., MNIST) often fail to generalize to unseen architectures (e.g., ResNet-50 or BERT), leading to suboptimal performance.

Methodology

The paper proposes AMORL (Adaptive Multi-Objective RL-CNN), a generalized framework that combines Reinforcement Learning (RL) and Convolutional Neural Networks (CNN) to create a scalable, adaptive mapping policy for heterogeneous NoC architectures.

Core Architecture

AMORL employs a hierarchical learning structure:

RL-CNN Policy Network: The core agent uses a CNN to process spatial state information (normalized compute demands, communication intensity matrices, and core utilization) to detect spatial patterns, such as communication hotspots and underutilized regions. The CNN outputs a probability map guiding the RL agent's decision on which core to assign to a specific neuron cluster. The RL agent optimizes the temporal sequence of these assignments.
Meta-Controller: A secondary neural network that dynamically weights five distinct objectives—communication cost, load imbalance, energy efficiency, latency, and thermal risk—based on the current system state. It outputs a softmax-weighted vector to combine these objectives into a single scalar reward, allowing the system to prioritize different goals (e.g., thermal safety vs. latency) depending on runtime conditions.
Adaptation Module: To handle sudden workload shifts (e.g., new batch sizes or model types), the framework extracts fixed-size workload patterns via adaptive pooling. Upon detecting a change, it triggers a fast online adaptation step using top-k similar past experiences to fine-tune a small adaptation vector, enabling specialization without full retraining.

Training-Aware Mapping

A key innovation is the framework's ability to distinguish between DNN training phases. The state representation includes one-hot flags for forward, backward, and weight-update phases. The reward function applies phase-specific penalties (e.g., doubling the communication cost penalty during the backward pass, which dominates training time). This allows AMORL to optimize placements specifically for gradient propagation and weight synchronization without altering the underlying DNN optimizer (e.g., SGD).

Workload and Environment

The framework was trained and evaluated on six diverse DNN workloads ranging from small, fully connected networks (IRIS, 3 layers) to large, deep architectures (ResNet-50, 53 layers) and transformer-based models (BERT-tiny). Training utilized a Multi-Dataset paradigm, interleaving episodes from all six workloads to learn a robust, generalizable policy. Evaluations were conducted on NoC sizes ranging from 4×4 to 32×32 meshes.

Key Contributions

Hierarchical Learning Architecture: Introduction of a meta-controller that dynamically weights multi-objective rewards and an RL-CNN policy that learns spatial mapping patterns and temporal assignment sequences.
Training-Aware, Phase-Specific Mapping: Extension of the RL-CNN to encode training phases (forward, backward, update) in the state space, optimizing for the specific communication bottlenecks of each phase (e.g., minimizing gradient hops).
Fast Online Adaptation: A mechanism to detect workload shifts and perform rapid specialization using a few gradient steps on an adaptation vector, avoiding catastrophic forgetting and the need for retraining from scratch.
Multi-Objective Reward with Thermal Modeling: Integration of per-hop latency and a thermal-risk model into a 5-dimensional reward vector, enabling the framework to balance speed, efficiency, and reliability.
Scalability and Generalization: Demonstration of size-invariant policies trained on random mesh sizes (4×4 to 32×32) and diverse DNNs, achieving constant-time ( $O(1)$ ) action selection during inference.

Experimental Results

The framework was evaluated using an in-house DNN mapper and cycle-accurate simulations via gem5/Garnet.

Performance Improvements: Compared to ablated variants (removing the meta-controller, CNN, or adaptation module) and baseline algorithms (Random, Simulated Annealing, Genetic Algorithm), AMORL achieved:
- Communication Cost: Reduction of over 70% compared to ablated variants and up to 69% over Random baselines.
- Load Imbalance: Reduction of up to 86% as NoC size increased.
- Latency: Reduction of more than 50% compared to ablated variants, with peak savings of 78% for BERT-tiny on smaller NoCs.
- Energy: Consistent energy savings ranging from 2.5% to 6% over baselines.
Scalability: Load imbalance decreased monotonically with NoC size, reaching near-zero levels (0.031–0.035) on 32×32 meshes, indicating near-perfect load distribution.
Cycle-Accurate Validation:
- On a 4×4 NoC, AMORL sustained throughput up to 876 Gbps with stable latency under light loads.
- On an 8×8 NoC, throughput exceeded 2.77 Tbps, confirming robustness under realistic traffic conditions and varying injection rates (0.05 to 0.50 flits/node/cycle).
Training Dynamics: The meta-controller rapidly learned to balance objectives, reducing the weight of the numerically dominant energy objective while increasing the weight of latency. The policy converged to a stable plateau by episode 600.

Significance and Claims

The paper claims that AMORL provides a unified, adaptive, and multi-objective mapping solution that addresses the critical challenges of modern many-core DNN accelerators. Its significance lies in:

Generalization: Unlike prior works tailored to specific architectures, AMORL learns a single policy capable of handling diverse DNNs (from tiny to large) and NoC configurations without per-workload retraining.
Dynamic Adaptation: It effectively manages the dynamic nature of real-world workloads, including multi-model environments and varying training phases, through its online adaptation and phase-aware mechanisms.
Holistic Optimization: By jointly optimizing communication, load, energy, latency, and thermal safety, it offers a practical solution for energy-efficient and thermally safe DNN execution.
Validation: The results, validated through both analytical modeling and cycle-accurate simulation, confirm that the learned policies generalize beyond the training environment, maintaining efficiency and low latency under realistic traffic conditions.

The author concludes that AMORL represents a step forward in enabling efficient, scalable, and adaptive execution of both inference and training workloads on heterogeneous NoC-based many-core systems.