⚡ electrical engineering

TransfoREM: Transformer aided 3D Radio Environment Mapping

This paper proposes TransfoREM, a novel 3D Radio Environment Mapping method that leverages a transformer model to integrate deterministic channel models with real-world data, thereby improving interpolation accuracy for UAV connectivity and enabling enhanced resource allocation and interference management in terrestrial cellular networks.

Original authors: Gautham Reddy, Ismail Guvenc, Mihail L. Sichitiu, Arupjyoti Bhuyan, Bryton Petersen, Jason Abrahamson

Published 2026-01-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Gautham Reddy, Ismail Guvenc, Mihail L. Sichitiu, Arupjyoti Bhuyan, Bryton Petersen, Jason Abrahamson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out how strong a radio signal is for a drone flying high in the sky. The problem is that cell towers are built to talk to phones on the ground. They point their "voices" (signals) slightly downward. When a drone flies up, it often finds itself in a weird spot: it can see the tops of many towers at once, but the signals might be weak, jumbled, or blocked by the way the towers are tilted.

To fix this, the researchers in this paper created a new tool called TransfoREM. Think of it as a 3D weather map for radio signals. Instead of showing rain and wind, this map shows exactly how strong the cell signal is at every point in the sky around a cell tower.

Here is how they built it, using simple analogies:

1. The Problem: The "Blind Spot"

Traditional maps are like flat paper maps. They work great for driving on roads, but they fail for drones flying in 3D space. The researchers needed a way to predict the signal strength in the air without having to fly a drone to every single spot to measure it (which would take forever).

2. The Solution: Turning Signals into a Story

The core idea of TransfoREM is to treat radio signals like sentences in a book.

The Old Way: Imagine trying to guess the weather by looking at a single snapshot. It's hard to see the pattern.
The New Way (TransfoREM): The researchers realized that as you move away from a cell tower, the signal changes in a predictable pattern, like words in a sentence. They took the 3D space around the tower and turned it into a long list of "words" (signal strengths) that follow a specific order.

3. The Brain: A "Translator" AI

They used a type of Artificial Intelligence called a Transformer. You might know these from apps that translate languages (like English to Spanish).

How it works here: Instead of translating words, this AI translates distance and direction into signal strength.
The "Masking" Trick: Imagine you are learning a new language. You read a sentence with some words hidden (masked) and have to guess what they are. The AI does this too. It looks at the known signal strengths and has to guess the hidden ones. This helps it learn the "grammar" of how radio waves travel.

4. The Two-Step Training Process

The AI didn't just learn from scratch; it learned in two stages, like a student:

Stage 1: The Textbook Learning (Pre-training)
First, the AI studied a "textbook" (a physics model). It learned the basic rules of how radio waves travel in a perfect, empty world. It learned that signals get weaker as you get farther away, just like a voice gets quieter the further you walk from a speaker.
Stage 2: The Real-World Internship (Fine-tuning)
Textbooks aren't perfect. In the real world, buildings reflect signals, and antennas aren't perfect. So, the researchers gave the AI a small amount of real-world data (measurements taken by a drone). The AI then adjusted its "textbook knowledge" to match reality. It learned the specific quirks of that particular cell tower.

5. The Results: Better than the Competition

The researchers tested their new map against two other methods:

Kriging: A traditional statistical method (like drawing smooth lines between dots on a map).
TripleLayerML: A complex machine learning method that uses three different models stacked on top of each other.

The Findings:

Accuracy: TransfoREM was more accurate than the traditional Kriging method. It could guess the signal strength in empty spaces better than the old methods.
Efficiency: While the "TripleLayerML" method was also accurate, it was like building a massive, complicated factory to make a simple tool. TransfoREM was simpler, faster, and just as accurate after its second training stage.
3D Capability: Unlike the old methods that struggled to connect data from different heights, TransfoREM could easily understand the 3D shape of the signal, even when it had to guess (extrapolate) for altitudes it hadn't seen before.

Summary

In short, TransfoREM is a smart system that helps cell towers understand their own "voice" in the sky. By treating radio signals like a sequence of words and using a powerful AI translator, it creates a detailed 3D map of coverage. This helps network managers know exactly where drones can fly and stay connected, without needing to measure every single inch of the sky.

Technical Summary: TransfoREM – Transformer aided 3D Radio Environment Mapping

Problem Statement
Providing reliable cellular connectivity to Unmanned Aerial Vehicles (UAVs) remains a significant challenge because existing terrestrial networks are optimized for ground-level coverage. At higher altitudes, UAVs experience complex propagation conditions, including simultaneous line-of-sight (LOS) to multiple base stations (BSs) and varying antenna patterns caused by the downward tilt of terrestrial BSs. Traditional network planning relies on deterministic propagation models that often fail to capture real-world deviations like multipath and diffraction. Furthermore, purely data-driven Radio Environment Map (REM) reconstruction techniques often require dense sampling rates, which are difficult to achieve in 3D space. There is a need for a method that can proactively maintain a spatio-temporal database of the Radio Frequency (RF) environment to manage interference, optimize resource allocation, and support dynamic spectrum access, particularly for aerial users.

Methodology
The authors propose TransfoREM, a hybrid 3D REM generation framework that combines deterministic channel models with real-world data using a Transformer architecture. The core innovation lies in reframing radio propagation modeling as a sequence prediction task within a spherical coordinate system, aligning with the physical nature of outward radiation and antenna directivity.

The methodology proceeds through the following steps:

Coordinate Transformation: 3D spatial coordinates $[x, y, z]$ are transformed into spherical coordinates $[\rho, \phi, \theta]$ relative to the BS. The radial distance $\rho$ is discretized into bins, creating a sequence of signal strengths originating from the BS.
Feature Encoding: For each spatial point, a feature sequence $\Gamma_i$ is constructed. This vector encodes the 3D spatial direction and includes logarithmic radial steps, angular coordinates, and Cartesian coordinates along the specific angular direction.
Two-Stage Hybrid Training:
- Stage 1 (Model-based Pre-training): The Transformer encoder is pre-trained on synthesized data generated from a Free Space Path Loss (FSPL) model with characterized antenna gain patterns. This stage teaches the model the fundamental inverse square law of signal attenuation and antenna directivity. Random masking is applied to the input sequences during this phase.
- Stage 2 (Data-driven Fine-tuning): The model is fine-tuned using limited real-world Received Signal Received Power (RSRP) measurements. In this stage, masking is adjusted to reveal only the specific bin corresponding to the measurement point while hiding the rest of the sequence. The model learns to adapt to site-specific deviations, such as actual beam patterns and shadow fading, using a Smooth L1 loss function to handle noise and phase mismatches in the real data.

Key Contributions
The paper outlines three primary contributions:

Sequence-based Propagation Modeling: Radio propagation is formulated as a sequence estimation task in spherical coordinates, leveraging the inherent radial nature of wireless signal radiation.
Transformer Application for REMs: The authors demonstrate that a Transformer model, utilizing masked sequence translation techniques borrowed from natural language processing, can accurately predict radio propagation. This approach is shown to be effective even when real-world data is sparse.
3D REM Framework: The proposed method generates a true 3D REM that can extrapolate from both 2D sample measurement planes and 3D spatial data. It is designed for holistic integration at the individual Base Station level to learn local radio footprints.

Experimental Results
The authors evaluated TransfoREM using two datasets: the AERPAW dataset (RSRP measurements from 30m to 110m altitudes) and a dataset from prior work [2] containing LTE KPIs.

Performance Improvement: On the AERPAW dataset, the two-stage training approach significantly improved accuracy. The Root Mean Square Error (RMSE) decreased from 7.49 dB (Stage 1) to 4.57 dB (Stage 2), and the Coefficient of Determination ( $R^2$ ) improved from 0.33 to 0.77.
Comparison with Kriging: When compared against the Kriging algorithm (a standard interpolation method), TransfoREM demonstrated superior extrapolation capabilities across different altitudes. The Transformer-based approach achieved an average improvement of 1.5 dB in Median Absolute Error over Kriging when testing on disjoint altitude subsets.
Comparison with TripleLayerML: In a case study against a "TripleLayerML" approach (which uses linear regression, ensemble methods, and Gaussian Process Regression), TransfoREM achieved comparable performance after Stage 2 (RMSE of 1.29 dB vs. 1.27 dB for TripleLayerML). However, TransfoREM is noted to be less complex, avoiding the $O(N^3)$ computational complexity associated with the Gaussian Process layer in the TripleLayerML approach.

Significance and Claims
The paper claims that TransfoREM offers a robust solution for constructing 3D REMs by effectively bridging the gap between theoretical propagation models and sparse real-world measurements. By treating the problem as a sequence prediction task, the method captures spatial dependencies more effectively than traditional interpolation techniques like Kriging.

The authors emphasize that the framework is designed for deployment at the Base Station level, enabling the continuous, online learning of local radio propagation environments. This capability is intended to support enhanced resource allocation, interference management, and spatial spectrum utilization, particularly for serving aerial users in dynamic 3D spaces. The work distinguishes itself as a "physics-inspired" technique that learns foundational model knowledge before adapting to measurement data, rather than relying solely on data-driven reconstruction or purely deterministic modeling.