🔬 optics

Low-complexity neural network equalization for long-haul coherent transmission with cascaded semiconductor optical amplifiers

This paper numerically demonstrates that low-complexity neural networks can effectively compensate for accumulated distortions in long-haul coherent transmission systems using cascaded semiconductor optical amplifiers, achieving a significant bit error rate reduction in the low-dispersion O-band.

Original authors: S. Bogdanov, S. Sygletos, O. Sidelnikov, G. Gomes, M. Kamalian-Kopae, S. K. Turitsyn

Published 2026-03-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: S. Bogdanov, S. Sygletos, O. Sidelnikov, G. Gomes, M. Kamalian-Kopae, S. K. Turitsyn

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a delicate, high-speed message (like a video call or a massive file) across a very long distance using a network of fiber-optic cables. To keep the signal strong over hundreds of miles, you need to boost it up every few miles, kind of like a runner needing water stations.

In this paper, the researchers are testing a specific type of "water station" called a Semiconductor Optical Amplifier (SOA).

The Problem: The "Noisy, Shaky" Booster

SOAs are great because they are cheap, small, and can boost a huge range of colors (wavelengths) of light at once. However, they have a bad habit. When they boost the signal, they get "tired" and "shaky."

The Shaky Part: If the message comes in too fast, the booster can't recover quickly enough. It starts reacting to the previous part of the message while trying to boost the current part. This creates a "pattern effect," like a stuttering echo.
The Noisy Part: They also add a lot of static (noise) to the signal.
The Distortion: As the signal passes through a long chain (cascade) of these boosters, the message gets scrambled, twisted, and garbled. By the time it reaches the end, it's hard to understand.

The Solution: A "Smart Translator" (Neural Network)

To fix this mess, the researchers didn't try to build a perfect, expensive booster. Instead, they built a digital "Smart Translator" at the receiving end.

Think of the signal arriving at the receiver as a sentence written in a language that has been heavily distorted by a bad translator.

The Old Way: You might try to guess the meaning based on simple rules, but it's often wrong.
The New Way (Neural Network): They used a Neural Network (NN). Imagine a very smart, fast student who has read millions of examples of "distorted sentences" and knows exactly how to translate them back to the original meaning. This student looks at the garbled signal, recognizes the specific "stutter" and "twist" caused by the SOAs, and cleans it up instantly.

The Twist: The "Road Condition" Matters

The researchers discovered something fascinating about the road conditions (the fiber optic cable itself):

The "Windy" Road (High Dispersion): In standard fiber cables (C-band), the light spreads out and gets messy due to the physics of the glass. This "windy road" makes it very hard for the Smart Translator to do its job. Even with the translator, the message is still a bit fuzzy.
The "Smooth" Road (Low Dispersion/O-band): The researchers tested the system on a different type of light (O-band) where the road is incredibly smooth and the light doesn't spread out. On this smooth road, the Smart Translator worked miraculously well. It cleaned up the signal so effectively that the error rate dropped by a factor of 10 (an order of magnitude).

Why This Matters

Real-Time Speed: The "Smart Translator" they designed is very simple and lightweight. It's not a super-computer; it's a small, efficient program that could run on a chip in a router right now. This means we don't have to wait for future technology to use this.
Cost Savings: Because SOAs are cheap and small, if we can use this "Smart Translator" to fix their mistakes, we can build massive, ultra-wide internet networks without needing expensive, complex equipment.
The Future: This suggests that for certain types of high-speed internet connections (especially those that don't need to travel through the most "windy" parts of the fiber), we can use cheap amplifiers and just let a smart AI clean up the signal at the end.

In a nutshell: The paper shows that while cheap, noisy amplifiers (SOAs) mess up long-distance signals, a simple, fast "AI cleaner" can fix the mess almost perfectly—especially if the signal travels on a "smooth road" where it doesn't get naturally distorted by the cable itself.

1. Problem Statement

The paper addresses the challenges of deploying Semiconductor Optical Amplifiers (SOAs) in long-haul, high-capacity coherent optical communication systems. While SOAs offer significant advantages—such as cost-effectiveness, compact size, electrical pumping (no optical pumps required), and broad amplification bandwidth (suitable for O-band and ultra-wideband transmission)—they suffer from severe nonlinear impairments.

Key impairments include:

Gain Saturation and Pattern Effects: Due to finite carrier lifetimes, the gain does not fully recover between symbols, causing symbol-to-symbol power fluctuations.
Nonlinear Phase Noise: Temporal gain variations translate into phase noise via the linewidth enhancement factor ( $\alpha_H$ ).
Accumulation: In cascaded multi-span systems, these distortions accumulate, leading to substantial Bit Error Rate (BER) penalties.
Dispersion Interaction: The interaction between SOA nonlinearities and fiber chromatic dispersion complicates signal recovery, particularly in standard C-band transmission.

Existing mitigation strategies (e.g., linearizing drive currents or complex digital back-propagation) are often computationally expensive or require precise system knowledge, making real-time implementation difficult.

2. Methodology

The authors propose a low-complexity Neural Network (NN) equalizer designed for real-time implementation (e.g., on FPGAs) to compensate for accumulated distortions in a long-haul coherent link.

System Simulation:
- Transmission: A single-polarization, single-channel 16-QAM signal at 32 Gbaud.
- Link: 16 spans of 73 km standard single-mode fiber (SSMF), each followed by an inline SOA and a Gaussian optical filter.
- Modeling:
  - Fiber: Modeled using the Nonlinear Schrödinger Equation (NLSE) solved via the split-step Fourier method.
  - SOA: Modeled using a simplified dynamical rate equation approach (gain dynamics and ASE noise injection). Parameters were tuned to match a commercial Thorlabs SOA1013S.
- Receiver Processing: Linear dispersion compensation (LDC) $\rightarrow$ Matched RRC filter $\rightarrow$ Downsampling $\rightarrow$ NN Equalizer $\rightarrow$ Demodulation.
Neural Network Architecture:
- Type: Multilayer Perceptron (MLP).
- Input: A window of 33 consecutive complex-valued symbols (I and Q components concatenated).
- Structure: A baseline model with one hidden layer containing 128 neurons (tanh activation) and two outputs (predicting I and Q of the central symbol). Variants (17-64, 33-256) were also tested.
- Rationale: The architecture is intentionally kept shallow and simple to ensure feasibility for Field-Programmable Gate Array (FPGA) deployment, avoiding the latency of recurrent architectures like LSTMs.
- Training: Trained using TensorFlow 2.0 with Mean Squared Error (MSE) loss on $10^6$ symbols.

3. Key Contributions

Real-Time Feasibility: Demonstrated that a shallow MLP (comparable to existing FPGA implementations) can effectively equalize complex, cascaded SOA distortions without requiring deep, computationally heavy recurrent networks.
Dispersion-Dependent Performance: Identified a critical relationship between fiber dispersion and NN equalization efficacy. The study shows that while NNs struggle with high-dispersion channels (C-band), they achieve order-of-magnitude improvements in low-dispersion regimes (O-band).
Separation of Impairments: Systematically decoupled the effects of SOA nonlinearity from fiber nonlinearity and dispersion to quantify the specific contribution of the NN to each impairment type.

4. Key Results

The study evaluated the system under various conditions, comparing the NN-equalized system against a baseline without equalization and against systems using linear amplifiers (LinAmp).

Pure SOA Cascade (No Fiber):
- When fiber spans were replaced with attenuators (removing dispersion and fiber nonlinearity), the NN equalizer achieved a 57x reduction in BER. This confirms the NN's ability to effectively learn and compensate for pure SOA-induced nonlinearities.
Full System (SOA + Fiber):
- In the complete 16-span system, the NN provided a 3x reduction in BER. The performance gain was lower here because the joint action of SOA nonlinearity and fiber dispersion creates a more complex channel memory that a simple MLP struggles to fully invert.
Impact of Dispersion ( $\beta_2$ ):
- C-band (High Dispersion, $\beta_2 \approx -21.7$ ps $^2$ /km): The NN provided a 3x BER reduction.
- O-band/Low Dispersion ( $\beta_2 \approx -1$ ps $^2$ /km): The NN provided a 9x BER reduction for the SOA system and up to a 36x reduction for the linear amplifier system.
- Conclusion: As dispersion approaches zero, channel memory is reduced, allowing the low-complexity NN to operate with significantly higher efficiency.
Robustness: The results remained consistent across different NN architectures (17-64, 33-256), suggesting that the specific hyperparameters are less critical than the general suitability of the MLP for this task.

5. Significance and Conclusion

This paper establishes that low-complexity neural networks are a viable solution for long-haul coherent transmission using cascaded SOAs, provided the system operates in a low-dispersion regime (such as the O-band).

Practical Implication: This finding supports the use of SOAs in cost-effective, ultra-wideband, and data-center interconnect applications where the O-band is preferred. It suggests that the "memory" introduced by chromatic dispersion is the primary bottleneck for simple NN equalizers, not the SOA nonlinearity itself.
Future Outlook: The work paves the way for real-time, FPGA-based DSP solutions that can unlock the potential of SOAs in long-haul networks, offering a compact and energy-efficient alternative to traditional Erbium-Doped Fiber Amplifiers (EDFAs) and complex multi-stage amplification schemes.