🤖 machine learning

Forward-Forward Autoencoder Architectures for Energy-Efficient Wireless Communications

This paper proposes and evaluates Forward-Forward autoencoder architectures for energy-efficient wireless communications, demonstrating their competitiveness with backpropagation-trained systems in various channel scenarios while offering significant savings in memory and processing time.

Original authors: Daniel Seifert, Onur Günlü, Rafael F. Schaefer

Published 2026-02-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Daniel Seifert, Onur Günlü, Rafael F. Schaefer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to send secret messages across a stormy ocean. The robot has two parts: a Sender (Encoder) who writes the message, and a Receiver (Decoder) who tries to read it after the waves (noise) have scrambled it.

For decades, the standard way to teach this robot was called Backpropagation (BP). Think of BP like a very strict, slow teacher who works in a specific way:

The robot tries to send a message.
The Receiver tries to read it and finds mistakes.
The Receiver sends a "correction note" all the way back to the Sender, layer by layer, saying, "You did this wrong, fix it."
The Problem: The Sender has to wait for the Receiver to finish checking everything before it can learn. Also, the teacher needs to remember every single tiny detail of the journey to write these correction notes. This takes a lot of brainpower (memory) and energy, and it breaks if the ocean (the channel) is too chaotic to measure precisely.

The New Idea: Forward-Forward (FF) Learning

This paper introduces a new, more efficient way to teach the robot called Forward-Forward (FF) learning. Instead of a strict teacher sending notes backward, imagine a Coach who uses a "Good vs. Bad" game.

Here is how the FF Coach works:

1. The "Good" and "Bad" Drills

Instead of looking at the final error and working backward, the Coach runs two separate drills for every layer of the robot:

The "Good" Drill (Positive Pass): The Coach shows the robot a message that should work (e.g., "Send the word 'Hello' and receive 'Hello'"). The robot tries to make its internal activity feel "good" (energetic and confident).
The "Bad" Drill (Negative Pass): The Coach shows the robot a message that shouldn't work (e.g., "Send 'Hello' but pretend you received 'Goodbye'"). The robot tries to make its internal activity feel "bad" (weak or confused).

The robot learns by simply trying to maximize the "Good" feeling and minimize the "Bad" feeling at every single step, without needing to wait for a final verdict from the end.

2. No More "Backward Lock"

In the old method (BP), the whole team had to stand still and wait for the last person to finish grading before the first person could move. It was a traffic jam.
In the FF method, every layer of the robot can learn independently and simultaneously. It's like a relay race where every runner practices their own leg of the race at the same time, rather than waiting for the finish line to see how they did. This makes training much faster and allows for "pipelining" (doing things in a continuous flow).

3. Handling the "Un-teachable" Ocean

Sometimes, the ocean is so rough that you can't mathematically describe the waves (non-differentiable channels). The old BP teacher gets stuck because it can't calculate the "slope" of the error.
The FF Coach doesn't care about slopes. It just asks: "Did this look like a good message or a bad message?" This allows the robot to learn even when the communication channel is messy or involves hard, unchangeable steps (like forcing the signal to be a specific type of digital switch, known as quantization).

What Did the Researchers Find?

The researchers built this new robot and tested it against the old one:

It Works Just as Well: In normal conditions (calm or stormy seas), the FF robot performed almost as well as the old BP robot.
It Wins on Hard Tasks: When they forced the robot to use "hard" digital switches (which the old teacher hates), the FF robot actually outperformed the old one. The old teacher struggled to learn through the switches, but the FF Coach just kept playing the "Good vs. Bad" game and figured it out.
It Saves Energy and Memory: Because the FF robot doesn't need to store a massive list of "correction notes" for every single step, it uses much less memory. This is huge for battery-powered devices (like your phone or a sensor in a remote field).
It's Faster: Since layers don't have to wait for each other, the training process is much more efficient.

The Big Picture

Think of Backpropagation as a high-end, expensive supercomputer that needs a perfect map of the world to learn. It's powerful but heavy and slow.

Think of Forward-Forward as a clever, low-power drone that learns by trial and error in real-time. It doesn't need a perfect map; it just needs to know what feels "right" and what feels "wrong."

Why does this matter?
As we move toward 6G and the "Internet of Things," we need communication systems that are incredibly energy-efficient and can work in unpredictable environments. This paper shows that we can build these systems using a new learning style that is lighter, faster, and doesn't get stuck when things get messy. It's a step toward making our wireless networks smarter and greener.

1. Problem Statement

The application of Deep Learning (DL) to wireless communications, specifically for end-to-end learned coding and modulation, has traditionally relied on the Backpropagation (BP) algorithm. However, BP faces significant hurdles in practical communication systems:

Differentiability Requirement: BP requires a fully differentiable path through the entire network, including the channel. Real-world channels are often non-differentiable or only observable via input/output samples, necessitating simplified models or complex workarounds (e.g., Generative Adversarial Networks or Reinforcement Learning with feedback links).
Memory and Energy Inefficiency: BP requires storing partial derivatives for every node during the backward pass. This creates a "backward lock," where layers must wait for gradients from subsequent layers, hindering pipelining. This is particularly problematic for neuromorphic and analog hardware implementations (e.g., memristors) where storing derivatives is energy-intensive and physically difficult.
Hardware Constraints: Current hardware implementations often rely on external training (ex-situ) followed by weight mapping, which suffers from performance degradation due to hardware imperfections. True in-situ training is difficult with BP due to the backward pass requirements.

2. Methodology

The authors propose replacing BP with the Forward-Forward (FF) algorithm, a learning paradigm inspired by biological plausibility, to train end-to-end autoencoders for wireless communications.

Architecture

Model: An autoencoder consisting of an Encoder and a Decoder, both composed of Fully Connected (FC) layers with ReLU activations (except the final layer).
Channel: The system is tested over Additive White Gaussian Noise (AWGN) and Rayleigh Block Fading (RBF) channels.
Non-Differentiable Handling: The architecture supports scenarios where the encoder output is quantized (e.g., to BPSK symbols), a non-differentiable operation that breaks standard BP.

The Forward-Forward Training Mechanism

Unlike BP, which uses a forward pass followed by a backward pass, FF performs two forward passes per training step:

Positive Pass: The network processes "positive" data (the true message $m$ concatenated with itself). The goal is to maximize a "goodness" metric ( $g_i = \|a_i\|^2$ ) at each layer.
Negative Pass: The network processes "negative" data (the true message $m$ concatenated with a random, different message $\bar{m}$ ). The goal is to minimize the goodness metric.
Loss Function: Each layer $i$ $i$ has an individual loss function based on a threshold $\tau_i$ $τ_{i}$ :
- $L_i = \zeta(-(g_i - \tau_i))$ for positive data (encourages high activity).
- $L_i = \zeta(g_i - \tau_i)$ for negative data (encourages low activity).
- Here, $\zeta$ is the softplus function.
Classifier: A separate classification layer (with Softmax) is trained using standard Cross-Entropy Loss on "neutral" data (message $m$ concatenated with zeros) to map decoder activities to the original message.

Input Construction

Positive: $v = (1_m || 1_m)$
Negative: $v = (1_m || 1_{\bar{m}})$ where $\bar{m} \neq m$
Neutral (Inference): $v = (1_m || 0)$

3. Key Contributions

First FF-based Communication Autoencoder: The paper designs and numerically evaluates end-to-end learned autoencoders trained exclusively with the FF algorithm for wireless channels.
Handling Non-Differentiable Channels: The authors demonstrate that FF naturally handles non-differentiable operations (like hard quantization for BPSK) without requiring surrogate gradients (like the Straight-Through Estimator used in BP) or feedback links.
Performance Benchmarking: A comprehensive comparison is made against BP-trained systems and Reinforcement Learning (RL) approaches (which use noiseless feedback) across AWGN and RBF channels.
Resource Analysis: The paper quantifies the trade-offs, showing that while FF may require larger network widths for comparable performance, it offers significant advantages in processing time (pipelining) and memory usage (no gradient storage).

4. Key Results

Performance in AWGN: In continuous-output scenarios, FF autoencoders achieve Block Error Rates (BLER) close to BP-trained systems, with a slight performance gap (~1 dB) at high SNR.
Performance in RBF: FF systems compete effectively with BP and RL-based systems in Rayleigh fading environments.
Quantized Outputs (BPSK): This is the most significant finding. When the encoder output is forced to be non-differentiable (BPSK):
- BP systems suffer performance degradation unless using surrogate gradients (STE), which imperfectly approximate the gradient.
- BP-RL systems can close the gap but require complex feedback mechanisms.
- FF systems outperform both BP and BP-RL approaches in this scenario because the algorithm does not rely on backpropagating gradients through the quantizer.
Convergence: FF networks converge as fast as BP and significantly faster than RL-based approaches.
Network Size: FF networks generally require wider layers (higher capacity) to match the performance of BP networks. However, increasing depth alone does not yield the same benefits as it does for BP.
Efficiency:
- Time: FF reduces processing time by eliminating the "backward lock." For an $N$ -layer network, BP takes $2N$ time units, while FF takes $N+1$ .
- Memory: FF eliminates the need to store gradients for the backward pass, drastically reducing memory footprint and data traffic, which is critical for energy-efficient edge devices.

5. Significance

This work establishes the Forward-Forward algorithm as a viable, energy-efficient alternative to Backpropagation for wireless communication systems. Its significance lies in:

Enabling Analog/Neuromorphic Hardware: By removing the need for backward gradient storage and differentiable channels, FF makes in-situ training on low-power, analog hardware (like memristors) feasible.
Robustness to Real-World Constraints: It solves the "non-differentiable channel" problem naturally, allowing for the direct training of systems with hard constraints (e.g., fixed modulation schemes) without complex approximations.
Scalability: The ability to pipeline training (processing layers independently without waiting for global gradients) offers a pathway to faster training and deployment on resource-constrained devices.

In conclusion, while FF may require larger network architectures to match BP's raw accuracy in ideal conditions, its superior handling of non-differentiable constraints and its massive gains in memory and energy efficiency make it a promising candidate for next-generation, energy-efficient wireless communication systems.