⚡ electrical engineering

Multi-layer MIMO Relay as Deep Physical Neural Networks: Power Amplifiers as Activation Functions

This paper proposes a deep wireless physical neural network that utilizes multi-hop MIMO relays, where power amplifier nonlinearities serve as activation functions, enabling end-to-end trainable over-the-air inference for tasks like image classification with demonstrated performance benefits from hardware nonlinearity.

Original authors: Meng Hua, Itsik Bergel, Deniz Gündüz

Published 2026-07-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Meng Hua, Itsik Bergel, Deniz Gündüz

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a giant, bustling city where data is the traffic. Right now, most of this traffic is handled by massive, energy-hungry data centers—think of them as huge, air-conditioned factories where computers crunch numbers one by one. This works, but it's slow and eats up a lot of electricity. Scientists are now asking: what if we could stop sending the raw data to the factory and instead let the "roads" themselves do the thinking? This idea is called a "Physical Neural Network." Instead of digital code, these networks use the natural physics of the world—like light, sound, or radio waves—to perform calculations instantly. It's like if the road itself could decide which cars to speed up and which to slow down, rather than waiting for a traffic light controller in a distant office.

The specific corner of science this paper explores is "Wireless Physical Neural Networks" (WPNNs). In simple terms, this is about using radio waves to carry out the math behind Artificial Intelligence (AI). Usually, AI needs to multiply numbers and then squish the results through a special curve (called an "activation function") to learn patterns, like recognizing a cat in a photo. In a normal computer, this happens in digital chips. But in a wireless network, the radio waves naturally mix together, and the equipment that boosts the signal (called a Power Amplifier) has a quirk: it gets a little "squishy" or nonlinear when the signal gets too strong. For a long time, engineers thought this squishiness was a bug—a mistake that ruined the signal. This paper, however, suggests that if you design the system just right, that "bug" might actually be a superpower, turning the radio waves into a deep, thinking brain right in the air.

The authors of this paper, Meng Hua, Itsik Bergel, and Deniz Gündüz, propose a new way to build these wireless brains using a chain of relay stations. Imagine a game of "telephone," but instead of whispering a message, a group of relays are passing a complex image from a source to a user. In their setup, each relay station doesn't just shout the message louder; it acts like a layer in a neural network. The relay takes the incoming signal, applies a specific "gain" (like turning a volume knob) and a "bias" (like a slight offset), and then passes it through its own Power Amplifier. The paper treats the natural, nonlinear "squish" of that amplifier as the brain's activation function—the part that decides if a neuron should fire or not. By chaining many of these relays together, they create a "deep" network that processes information entirely over the air, without needing to convert the signal back and forth between digital and analog formats.

To test this idea, the researchers simulated a system trying to recognize images from the Fashion-MNIST dataset (a collection of 28x28 pixel pictures of clothes like shirts, boots, and bags). They compared two different ways of managing the radio signals. The first method, called "LS" (Least Squares), only requires the receiving end to know the state of the channel. The second, "SVD" (Singular Value Decomposition), requires both the sender and receiver to know the channel perfectly. The results were surprising and counter-intuitive. When the researchers used the "LS" method, the nonlinear "squishy" amplifiers actually helped the system get smarter as they added more relays. The more layers they added, the better the image recognition became, with the nonlinear system reaching an accuracy of about 92.6% with five relays, nearly matching a perfect digital computer.

However, the paper explicitly rules out the idea that this works for every setup. When they used the "SVD" method, which tries to perfectly separate the signal paths, the nonlinear amplifiers became a problem. The "squishiness" messed up the delicate separation of the signals, causing the performance to get worse as more relays were added. This tells us that the "magic" of the nonlinear hardware isn't a universal fix; it depends entirely on how the system is designed. Furthermore, the paper notes that if you train the system assuming the amplifiers are perfect and linear, but then test them with real, nonlinear amplifiers, the performance drops significantly, especially as the chain gets longer. This suggests that for these wireless brains to work in the real world, the hardware's quirks must be part of the training process, not ignored.

In these simulations, the authors found that the "LS" approach with nonlinear amplifiers is the winner for deep networks, turning a traditional hardware limitation into a feature that boosts the network's ability to learn complex patterns. They didn't just suggest this; they showed through their computer models that the accuracy improved steadily with the number of relays, provided the training accounted for the hardware's real behavior. The paper concludes that while this architecture is promising, future work needs to figure out how to make these systems robust against the messy realities of imperfect hardware and uncertain signal conditions. For now, the study shows that by embracing the natural nonlinearity of our radio equipment, we might be able to build AI that thinks as it travels, right over our heads.

Technical Summary: Multi-layer MIMO Relay as Deep Physical Neural Networks

Problem Statement
Conventional deep learning relies on digital GPUs, which suffer from the von Neumann bottleneck where the physical separation of memory and computation incurs significant energy and latency costs. While Physical Neural Networks (PNNs) offer an alternative by embedding computation into analog hardware, existing Wireless PNN (WPNN) implementations face limitations. Previous studies utilizing Reconfigurable Intelligent Surfaces (RISs) or relays have not fully exploited spatial degrees of freedom or the intrinsic nonlinearity of physical devices, thereby limiting their expressive capacity. Furthermore, the role of power amplifier (PA) nonlinearity in over-the-air computing has been underutilized, often viewed merely as a distortion to be mitigated rather than a computational resource.

Methodology
The paper proposes a deep WPNN architecture realized through a multi-hop Multiple-Input Multiple-Output (MIMO) relay network. In this system, information flows from a source to a user through $M$ relays. The core innovation lies in mapping the components of a digital neural network to physical layer operations:

Linear Mappings: Realized by trainable complex linear gains (amplification matrices) and biases at each relay.
Nonlinear Activations: Realized by the intrinsic nonlinearity of the Power Amplifiers (PAs) at each relay.

The system models the wireless channel using a Rician fading model. The authors develop two distinct transceiver designs based on Channel State Information (CSI) availability:

LS-based Scheme (CSIR): Requires only receiver-side CSI. Relays use a Least Squares (LS) estimator to decouple signals, followed by trainable amplification, bias addition, and the nonlinear PA activation. This architecture treats the cascade of relays as an over-the-air fully connected (FC) network.
SVD-based Scheme (CSIR/T): Requires both transmitter and receiver CSI. This design utilizes Singular Value Decomposition (SVD) to decouple spatial eigenmodes at each hop. The signal processing involves receive combining, singular-value equalization, amplification, and transmit pre-processing aligned with the next hop's channel.

The network is trained end-to-end using standard backpropagation through a differentiable PA model (specifically the Rapp model) to minimize a task-specific loss function (cross-entropy for image classification).

Key Contributions

Architecture Proposal: The authors formally associate the trainable amplification, bias, and PA nonlinearity at each relay with the weight, bias, and activation function of a fully connected layer, creating a deep over-the-air neural network.
Transceiver Designs: Two specific schemes are developed for different CSI scenarios (LS-based for CSIR and SVD-based for CSIR/T), both supporting end-to-end training.
Role of Nonlinearity: The paper investigates the impact of PA nonlinearity, demonstrating that it acts as a structural enabler for deep learning in the analog domain, breaking the rank limitations inherent in linear-only cascades.

Results
Simulations were conducted on the Fashion-MNIST dataset (10-class image classification) with varying Signal-to-Noise Ratios (SNR), numbers of relays ( $M$ ), and Rician factors ( $K$ ).

Nonlinearity vs. Linearity: Contrary to the expectation that nonlinearity is purely detrimental, the "LS, NPA" (Nonlinear PA) scheme significantly outperforms its linear counterpart ("LS, LPA") and approaches the performance of an ideal digital upper bound. The accuracy of the LS-based scheme with nonlinear PAs increases monotonically with the number of relays ( $M$ ).
Architecture Dependency: The benefits of nonlinearity are architecture-dependent. In the SVD-based scheme, the nonlinear PA disrupts the eigenmode decoupling, causing residual inter-stream interference. Consequently, the "SVD, NPA" scheme degrades as $M$ increases, whereas the "SVD, LPA" (Linear PA) scheme maintains consistent performance.
Training-Testing Mismatch: When a network trained assuming linear PAs is tested with nonlinear PAs (TPM), performance degrades significantly, and this error accumulates with the number of hops. However, training with lower power constraints (keeping PAs closer to the linear region) can mitigate this mismatch.
Channel Conditions: Performance generally degrades as the Rician factor $K$ increases (indicating lower-rank channels), as this reduces the spatial degrees of freedom essential for the WPNN's operation.

Significance and Claims
The paper claims that the proposed architecture successfully unifies communication and computation over the same wireless infrastructure, enabling accurate over-the-air inference. A primary finding is that PA nonlinearity should be viewed as a resource rather than an impairment for over-the-air computing, provided the system architecture (specifically the LS-based approach) is designed to exploit it. The study demonstrates that a multi-hop MIMO relay system can function as a deep physical neural network, offering a pathway to lower energy consumption and latency compared to digital implementations. The authors conclude that while the current work focuses on fully connected layers, the framework is extensible to more complex architectures like convolutional or attention-based WPNNs, and future work should address robust training under hardware model mismatches and CSI uncertainty.

More like this