⚡ electrical engineering

Beam Alignment in Multipath Environments for Integrated Sensing and Communication using Bandit Learning

This paper proposes an Integrated Sensing and Communication (ISAC) approach that leverages radar information to prune candidate beams in a Multi-Armed Bandit (MAB) framework, significantly reducing beam alignment exploration time and increasing communication throughput in multipath environments.

Original authors: Akanksha Sneh, Shobha Sundar Ram, Sumit J Darak, Aakanksha Tewari

Published 2026-02-10

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Akanksha Sneh, Shobha Sundar Ram, Sumit J Darak, Aakanksha Tewari

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are at a massive, crowded music festival held in a dark, sprawling field. You are a security guard (the Base Station) trying to shine a high-powered spotlight (the Communication Beam) on a specific friend (the Mobile User) so you can pass them a message.

The problem? The field is huge, and you have 41 different narrow spotlights. If you just start scanning every single spot in the field one by one to find your friend, you’ll waste so much time looking at empty grass or trees that by the time you find them, the moment has passed. This "wasted searching time" is what engineers call Exploration Time.

This paper proposes a smarter way to do this using two technologies working together: Radar and Smart Learning.

1. The "Radar" Assistant (The Scout)

Instead of just blindly swinging the spotlight around, the researchers suggest using Integrated Sensing and Communication (ISAC).

Think of this like having a small, high-speed motion sensor (the Radar) that stays on all the time. Before you even touch the big, heavy spotlight, the motion sensor "pings" the area. It doesn't tell you exactly who is there, but it shouts, "Hey! There’s something moving near the snack bar and the stage!"

Now, instead of searching all 41 spots in the field, you only focus your spotlight on the 5 or 6 spots where the sensor detected movement. You’ve just skipped the "searching the empty grass" phase entirely.

2. The "Bandit Learning" (The Memory)

The second part of the paper uses something called a Multi-Armed Bandit (MAB) algorithm.

Imagine you are at a casino with 41 slot machines (the "arms" of the bandit). You want to find the one machine that pays out the most (the Optimal Beam).

The Old Way: You play every single machine once to see what happens. If there are hundreds of machines, you'll go broke before you find the winner.
The Paper's Way: You use the Radar Scout to tell you which machines are even plugged in and moving. You only play those. Then, you use a mathematical formula (called UCB) to balance two things: playing the machine that has been winning lately, but occasionally checking other "promising" machines just in case they've improved.

3. Dealing with "The Moving Target" (The Chase)

The paper also tackles a tricky problem: What if your friend starts walking?

In a normal system, if your friend walks out of your spotlight, you might not realize it until the connection drops and you have to start the whole "search everything" process from scratch.

The researchers created a "Change Detection" system. Because the Radar is constantly watching the speed and direction of the movement, the system can actually predict when your friend is about to walk out of the light. It’s like a GPS that says, "He's walking left at 3 mph; in exactly 10 seconds, he'll be out of the beam. Get ready to move the light now!"

The Result: Why does this matter?

By combining the "Scout" (Radar) with the "Smart Gambler" (Bandit Learning), the researchers proved that:

Less Wasted Time: They reduced the "searching" time by 35%.
More Data: Because they spend less time searching and more time actually talking, the amount of information sent (the Throughput) increased by 1.4 times.
Hardware Ready: They didn't just do this on a computer; they proved it works on actual microchips (SoC) that could be put into real-world 6G towers or self-driving cars.

In short: It’s the difference between searching a dark room with a slow, heavy flashlight by scanning every inch of the floor, versus using a motion sensor to point your light exactly where the action is happening.

Technical Summary: Beam Alignment in Multipath Environments for ISAC using Bandit Learning

1. Problem Statement

In millimeter-wave (mmW) communications (5G/6G), high data rates and high connection densities necessitate the use of narrow, directional beams. However, maintaining these beams is challenging due to high mobility, significant Doppler shifts, and multipath propagation.

Existing solutions often rely on Multi-Armed Bandit (MAB) algorithms, specifically the Upper Confidence Bound (UCB) approach, to select optimal beams. The primary limitation of conventional MAB is the long exploration time: when the number of candidate beams ( $K$ ) is large, the algorithm must scan many sub-optimal beams to find the best one, leading to high "regret" (lost throughput) and high latency.

2. Methodology

The authors propose augmenting the UCB algorithm with Integrated Sensing and Communication (ISAC). The core premise is that the radar and communication functions share the same field-of-view, hardware, and spectrum (via Time Division Multiplexing).

A. Radar-Enhanced Beam Selection (Stationary Case):
Instead of performing a "round-robin" scan of all $K$ beams, the system first uses a radar waveform (extracted from the IEEE 802.11ad preamble) to detect mobile targets.

Radar Signal Processing (RSP): The system uses matched filtering for range estimation and the MUSIC algorithm for Doppler velocity estimation.
Beam Pruning: The radar identifies a much smaller subset of candidate beams ( $\tilde{K}$ ) where mobile targets are actually present. The UCB algorithm then only explores these $\tilde{K}$ beams, significantly reducing the exploration phase.

B. Change Detection (Quasi-Stationary Case):
In dynamic environments where a user moves out of a beam, the system must decide when to restart the beam alignment. Rather than using periodic restarts or complex statistical monitoring, the authors use RSP-based geometry. By using the estimated range ( $r_b$ ) and Doppler velocity ( $v_b$ ), the system calculates the expected time ( $T_\infty$ ) until the target leaves the current beam's width, triggering a realignment only when necessary.

C. Hardware-Software Co-Design (HSCD):
To ensure real-time feasibility, the authors implemented the RSP and MAB algorithms on a Zynq UltraScale+ SoC (FPGA + ARM processor). They utilized fixed-point analysis to optimize resource utilization and execution speed.

3. Key Contributions

ISAC-MAB Framework: A novel integration of radar sensing to prune the search space of reinforcement learning-based beam selection.
Predictive Realignment: A computationally efficient change-detection mechanism based on target kinematics (range and Doppler) rather than communication metrics.
Hardware Validation: A complete hardware-software co-design that demonstrates the practical feasibility of running complex RSP and MAB algorithms on edge devices.
Robustness to Multipath/Clutter: The methodology specifically addresses complex environments involving static clutter and multiple scattering paths.

4. Results

The proposed UCBISAC algorithm was evaluated against conventional MAB (UCBSNR/LUCB) and Digital Beamforming (DBF) through extensive simulations and hardware timing analysis:

Exploration Time: Achieved a 35% reduction in overall exploration time compared to conventional MAB.
Throughput: Demonstrated a 1.4x higher throughput factor compared to communication-only MAB.
Error Rates: Showed significantly lower Cumulative Bit Error Rates (BER) across various target models (point targets, pedestrians, and mid-size cars).
Clutter Rejection: The radar component successfully identified and eliminated beams corresponding to static clutter (zero Doppler), which typically degrade conventional MAB performance.
Hardware Efficiency: The FPGA-accelerated RSP achieved an execution time of ~3.5 ms, making it viable for the 4 ms time-slot requirements of mmW standards.

5. Significance

This work provides a critical bridge between theoretical reinforcement learning and practical mmW hardware implementation. By leveraging the "side information" provided by radar sensing, the authors solve the scalability issue of MAB in high-dimensional beam spaces. This approach is highly significant for the development of 6G vehicular communications (V2X) and industrial robotics, where rapid, reliable beam tracking in highly dynamic and cluttered environments is mandatory.