🔬 optics

Photonic spiking reinforcement learning for intelligent routing

This paper proposes and experimentally validates a photonic spiking reinforcement learning architecture based on proximal policy optimization that achieves ultra-low latency and high energy efficiency for intelligent routing in software-defined networks, significantly outperforming conventional algorithms while maintaining inference accuracy through a hardware-software collaborative framework.

Original authors: Shuiying Xiang, Yonghang Chen, Ling Zheng, Zhicong Tu, Xintao Zeng, Mengting Yu, Shuai Wang, Yahui Zhang, Xingxing Guo, Weitao Pan, Yue Hao

Published 2026-02-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Shuiying Xiang, Yonghang Chen, Ling Zheng, Zhicong Tu, Xintao Zeng, Mengting Yu, Shuai Wang, Yahui Zhang, Xingxing Guo, Weitao Pan, Yue Hao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a massive, bustling city where millions of delivery trucks (data packets) are constantly trying to get from point A to point B. In the past, the traffic controllers used a simple, rigid rulebook: "Always take the shortest path." This worked fine when traffic was light, but when the city got crowded, everyone tried to squeeze onto the same few roads, causing massive gridlock, delays, and lost packages.

This paper introduces a smarter, faster, and more energy-efficient way to manage this traffic using a combination of Artificial Intelligence and Light-based Computing.

Here is the breakdown of their solution using simple analogies:

1. The Problem: The Old Rulebook vs. The Real World

Traditional routing (like the Dijkstra algorithm mentioned in the paper) is like a GPS that only knows the shortest distance. It doesn't "learn" from traffic jams or sudden accidents. It just follows a static map. When the network gets busy, this leads to congestion and slow speeds.

2. The Solution: A "Super-Intelligent" Traffic Cop

The researchers built a new system using Reinforcement Learning (RL). Think of this as a traffic cop who doesn't just follow a map but actually learns from experience.

How it learns: The cop tries different routes. If a route is fast and smooth, it gets a "good job" reward. If it causes a jam, it gets a "try again" penalty. Over time, it figures out the perfect strategy to keep traffic flowing, balancing the load so no single road gets overwhelmed.
The Algorithm: They used a specific learning method called PPO (Proximal Policy Optimization), which is like a very disciplined student that learns efficiently without making wild, unpredictable mistakes.

3. The Hardware: Switching from Bricks to Lightbulbs

Usually, teaching this "traffic cop" requires powerful computer chips that eat up a lot of electricity and can be a bit slow to think. The researchers wanted something faster and greener.

Spiking Neural Networks (SNNs): Instead of a standard computer brain that processes information like a steady stream of water, they used a "Spiking" brain. This mimics how real biological neurons work: they stay quiet until they get enough signal, then they "fire" a quick spike of energy. This is much more energy-efficient, like a light switch that only uses power when it clicks on, rather than a dimmer bulb that is always on.
Photonic Computing: To make this even faster, they built the brain out of light instead of electricity. They used tiny chips made of lasers and mirrors (Mach-Zehnder interferometers) to process information at the speed of light.
- The Analogy: Imagine the difference between sending a letter by mail (electronic chips) versus sending a laser beam through a fiber optic cable (photonic chips). The laser beam is instantaneous and doesn't generate heat.

4. The Experiment: The "Fat-Tree" City

They tested this system in a simulated city called a "fat-tree" network (a common layout for data centers).

The Test: They pitted their new "Light-Based Smart Cop" against the old "Rule-Based GPS" (Dijkstra).
The Results: The new system won every time.
- Throughput: It moved more data (more trucks got through).
- Latency: It was faster (less time stuck in traffic).
- Packet Loss: Fewer packages were lost or dropped.
- Load Balance: It kept traffic spread out evenly, preventing any single road from getting clogged.

5. The "Hardware-Software" Team-Up

One of the coolest parts of the paper is how they actually built it. They didn't just simulate it on a computer; they built a real physical chip.

The Process: First, they trained the AI in software (on a regular computer). Then, they "mapped" that brain onto their physical light-based chip.
The Challenge: Real hardware has tiny imperfections (like a slightly crooked mirror). To fix this, they used a "collaborative" approach: the software adjusted itself to match the quirks of the physical light chip.
The Result: The physical light chip made decisions with 99% to 100% accuracy, matching the software version almost perfectly. This proves that you can run complex AI routing decisions on tiny, super-fast, low-energy light chips.

Summary

In short, this paper shows how to build a super-fast, low-energy traffic controller for the internet using light and biological-style learning. It learns to avoid traffic jams better than old methods and does it all on a physical chip that uses light instead of electricity, making it perfect for the massive, fast-paced data centers of the future.

Technical Summary: Photonic Spiking Reinforcement Learning for Intelligent Routing

Problem Statement
Intelligent routing is critical for modern communication infrastructures, including data centers and future 6G networks. While Deep Reinforcement Learning (DRL) offers a promising paradigm for optimizing routing in dynamic, bursty traffic environments, its practical deployment is hindered by high energy consumption and decision latency inherent in conventional artificial neural networks (ANNs). Traditional routing algorithms, such as Dijkstra, rely on static metrics and lack the adaptability to learn complex, nonlinear traffic patterns, often leading to local congestion and inefficient resource utilization. There is a need for a routing solution that combines the learning capabilities of DRL with the ultra-low latency and high energy efficiency required for real-time network control.

Methodology
The authors propose a Photonic Spiking Reinforcement Learning (PSRL) architecture that integrates the Proximal Policy Optimization (PPO) algorithm with Photonic Spiking Neural Networks (PSNNs) within a Software-Defined Networking (SDN) framework.

Architecture: The system operates on a hierarchical SDN structure (data, control, and application planes). The data plane utilizes a fat-tree topology (initially $k=4$ ) with OpenFlow switches. The application plane hosts the intelligent decision-making engine.
Algorithm: The routing agent employs a PPO Actor-Critic framework.
- Actor Network: Implemented as a Spiking Neural Network (SNN) using Leaky Integrate-and-Fire (LIF) neurons. It processes network states (encoded as spike trains) to output routing actions (path selection). The network uses a surrogate gradient method (Zero-Inflated Firing) to enable backpropagation through non-differentiable spike functions.
- Critic Network: Implemented as a standard ANN to evaluate state values and assist in policy optimization.
- State & Action: The state space includes host indices and path features (utilization, hop count). The action space is a discrete path-selection vector, filtered by an action mask to ensure feasibility.
- Reward: A multi-objective reward function optimizes throughput, packet loss, latency, and load balancing.
Hardware-Software Collaboration: To bridge the gap between algorithmic simulation and physical deployment, the authors developed a collaborative framework:
1. Software Pre-training: The SNN is trained from scratch in simulation.
2. In-situ Photonic Training: The linear weight matrices of the hidden layers are mapped onto a photonic Mach-Zehnder Interferometer (MZI) mesh chip using the Stochastic Parallel Gradient Descent (SPGD) algorithm to optimize phase-shifter voltages.
3. Hardware-Aware Fine-tuning: The mapped weights are reloaded into the software model for fine-tuning to compensate for fabrication imperfections and system noise.
4. Collaborative Inference: The final inference involves a hybrid loop where nonlinear spiking activations are computed on a CPU-FPGA platform, while linear matrix-vector multiplications and nonlinear spiking neuron functions are executed on photonic chips (MZI mesh and DFB-SA laser arrays).

Key Contributions

Architecture and Algorithm Innovation: The development of an intelligent routing model where the policy network is an SNN, aligning the computational paradigm with photonic neuromorphic hardware. This is integrated into an SDN control plane and evaluated on a fat-tree topology.
Performance Improvement: Extensive simulations demonstrate that the photonic spiking PPO algorithm significantly outperforms the conventional Dijkstra algorithm across throughput, latency, packet loss rate, and load balance, particularly under high network loads.
Hardware-Software Collaborative Computing: The realization of a framework that maps the spiking Actor network onto PSNN hardware. Experimental validation on 640 state-action pairs confirms that the hardware-software collaborative framework maintains decision accuracy consistent with pure algorithmic implementation.

Results

Simulation Performance: In a fat-tree network ( $k=4$ ), the photonic spiking PPO achieved a throughput of nearly 6000 Mbps (vs. limited growth in Dijkstra) and maintained average delays below 20 ms, whereas Dijkstra's delay exceeded 80 ms at 200 Mbps baseline traffic. Packet loss rates were significantly lower for the proposed method.
Hardware Validation:
- Weight Mapping: The similarity between digital weights and hardware-mapped weights on the MZI mesh reached 0.84 after 2100 epochs of SPGD optimization.
- Inference Accuracy: Experimental results across three traffic rates (40, 80, and 120 Mbps) showed high fidelity. The layer-wise error rates were low (0.5371%, 0.4785%, and 0.3516% respectively), with final inference accuracies reaching 99.38% to 100%.
- Fine-tuning: After hardware-aware fine-tuning, the retrained model retained superior performance over Dijkstra despite a slight decrease in reward compared to the pure software baseline.
Scalability: The method demonstrated robustness when scaled to a 128×128 hidden-layer MZI mesh and a larger fat-tree topology ( $k=6$ , 54 hosts). In the $k=6$ scenario, the proposed method achieved a peak throughput of ~12,000 Mbps (double that of Dijkstra) while maintaining lower packet loss and delay.

Significance
The paper claims that this work establishes a novel paradigm for intelligent routing optimization characterized by ultra-low latency and high energy efficiency. By tightly integrating the PPO algorithm's learning capabilities with the physical advantages of photonic hardware, the approach addresses the limitations of both traditional static routing and energy-intensive DRL. The authors posit that this integration bridges the research gap between photonic neuromorphic computing and network routing technologies, offering a viable solution for real-time network optimization in large-scale data centers, computing networks, satellite Internet systems, and future 6G networks. The successful hardware-software collaborative inference demonstrates the feasibility of deploying photonic spiking agents in practical SDN control planes.