⚡ electrical engineering

Quantized IRS Phase Control under Limited Observation with Short-History State Stacking

This paper demonstrates that short-history state stacking enhances the temporal input representation of DDQN-family controllers, significantly improving spectral efficiency for quantized intelligent reflecting surface phase control under limited observation compared to standard methods and geometric heuristics.

Original authors: Hongzhang Guo, Heng Yang, Shanshan Li, Yuwei Kong, Zicheng Yue, Lei Zhang, Jiahan Xu, Sixue Liu, Fangwei Li

Published 2026-09-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hongzhang Guo, Heng Yang, Shanshan Li, Yuwei Kong, Zicheng Yue, Lei Zhang, Jiahan Xu, Sixue Liu, Fangwei Li

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the invisible landscape of modern wireless communication, signals travel from a base station to a smartphone, but they often struggle to reach their destination. Buildings, hills, and even the movement of people can block or weaken these signals, creating dead zones where data slows to a crawl. To solve this, engineers have developed a technology called an intelligent reflecting surface. Imagine a large, flat panel covered in thousands of tiny, programmable mirrors. Unlike a standard mirror that simply reflects light in one direction, this panel can be tuned electronically to steer radio waves precisely where they are needed, effectively turning a wall or a ceiling into a smart helper that boosts the signal.

However, controlling this panel is a complex task. The surface is made of many individual elements, and each one must be set to a specific angle to reflect the signal correctly. In the real world, these angles cannot be set with infinite precision; they are limited to a set of discrete steps, much like a dimmer switch that only has ten specific brightness levels rather than a smooth slide. Furthermore, the people using the network are constantly moving, and the controller managing the surface does not have a perfect, real-time map of the entire environment. It only sees a snapshot of where users are, what they are doing, and how well the connection worked a moment ago. The challenge is to figure out how to adjust the mirrors quickly and accurately enough to keep the connection strong, even when the controller is working with limited information and only a few seconds of memory.

Researchers at Shenyang University of Technology set out to solve this specific puzzle. They wanted to know if giving the controller a short history of recent events—instead of just a single snapshot of the current moment—would help it make better decisions. In their study, they compared different computer programs designed to control the surface. Some programs looked only at the current situation, while others were allowed to look back at the last four steps of the user's movement and the system's performance. They tested these programs in a simulated environment where users moved along paths similar to those recorded from real smartphones, and the system had to adjust the mirrors to maintain the best possible data speed.

The researchers found that looking back in time did indeed help. When the controller used a short history of four steps, it performed better than when it relied on a single snapshot. Specifically, for one type of learning algorithm, this approach improved the data speed by about 6.9 percent compared to the single-snapshot version. For another type, the improvement was smaller but still positive, at roughly 1.7 percent. The study also showed that while these learning programs were very good at adapting to changes, a simpler method that calculated the mirror angles directly based on the users' positions was still the most effective at maximizing speed in their specific test setup. The learning programs, however, were better at balancing the need for speed with the cost of constantly changing the settings, which is a crucial factor for real-world stability.

The team also explored how long this history should be. They tested looking back at one, two, four, eight, and sixteen steps. They discovered that more memory was not always better. While four steps provided a clear advantage, looking back eight or sixteen steps did not continue to improve performance and actually made the system more complex and slower to process. This suggests that there is a sweet spot where a little bit of memory helps the system understand the flow of movement without overloading it with unnecessary data. The results held true even when the researchers introduced random variations to simulate the unpredictable nature of radio waves, with the short-history approach showing benefits in most of the test scenarios.

Ultimately, this work demonstrates that for systems controlling smart surfaces in a mobile world, a feed-forward controller—one that does not have a complex internal memory state—can be significantly improved simply by feeding it a short sequence of recent observations. The study confirms that this method is a practical and compact way to enhance performance without requiring a complete overhaul of the control architecture. While the simple geometric method of calculating angles directly remains the top performer in terms of raw speed in these simulations, the short-history approach offers a robust, data-driven alternative that adapts well to changing conditions. The findings provide a clear blueprint for engineers designing the next generation of wireless networks, showing that a modest amount of historical context can make a smart surface much smarter.

Technical Summary: Quantized IRS Phase Control under Limited Observation with Short-History State Stacking

Problem Formulation
This paper addresses the control of Finite-Resolution Intelligent Reflecting Surfaces (IRS) in mobile wireless environments where the controller operates under limited observation. In the considered scenario, users move along planar trajectories, but the controller does not have access to a complete instantaneous decomposition of direct and cascaded channels before every phase update. Instead, the controller receives a compact observation vector comprising:

Current user geometry (positions).
User activity status (active-user mask).
The previously applied IRS configuration.
Aggregate link feedback (mean Spectral Efficiency, mean SINR, and effective channel magnitude).

The core challenge is that a single observation at time $t$ may alias distinct underlying temporal states (e.g., different motion directions or near-future link conditions). The paper investigates whether explicitly providing a short-history state stack (a sequence of recent observations) to feed-forward Deep Reinforcement Learning (DRL) controllers can improve performance in this partially observable setting, without altering the underlying controller architecture (e.g., adding recurrent layers) or the quantized action interface.

Methodology
The study employs a controlled comparative framework involving six methods evaluated under a deterministic path-synthesis feedback model and an auxiliary stochastic Rician channel extension.

System Model: The environment consists of a multi-antenna Base Station (BS), a passive finite-resolution IRS, and mobile users. The IRS action space is multi-discrete, where each of the $L$ elements selects a phase from a quantized alphabet of size $M_\phi$ .
Controller Families: The study compares four Deep Q-Network (DQN) variants against two non-learning baselines:
- Baselines: Fixed Random (non-adaptive) and a Geometric Heuristic (constructs phases directly from current path geometry).
- Learning Controllers:
  - DDQN & D3QN: Standard and Dueling Double Q-Networks using a single-step observation ( $H=1$ ).
  - SEQ-DDQN & SEQ-D3QN: The same network architectures but utilizing a short-history state stack of $H=4$ consecutive observations as input.
Input Representation: The stacked-input variants replace the single observation vector $s_t$ with a concatenated vector $\tilde{s}^{(H)}_t = [s_t; s_{t-1}; \dots; s_{t-H+1}]$ . This increases the input dimension linearly but retains the feed-forward network structure.
Training Protocol: All learning controllers share a unified training protocol involving experience replay, double-Q targets, and hard target network synchronization. The reward function combines spectral efficiency (SE) with a penalty for phase reconfiguration switching.

Key Contributions
The paper makes the following specific contributions:

Trajectory-Based Formulation: It establishes a planar formulation for finite-resolution IRS control with mobile users, defining a compact controller observation interface that excludes instantaneous CSI but includes geometry and aggregate feedback.
Isolation of Temporal Input Effects: By instantiating short-history stacking in both DDQN and D3QN while keeping the action interface and network family fixed, the study isolates the performance gain attributable solely to explicit temporal input representation, distinct from recurrent or attention-based architectural changes.
Controlled Six-Method Comparison: It provides a rigorous comparison between non-adaptive, geometry-based, and feedback-driven value controllers, distinguishing between gains from temporal input and the inherent capabilities of value-based learning versus geometric construction.
Trade-off Characterization: The paper analyzes the balance between observation span, spectral efficiency, reward behavior, input dimension, and replay storage costs across different sequence lengths and channel profiles.

Results
In the reference configuration (4 users, 5x5 IRS grid, 4 phase levels):

Performance Gains: Short-history stacking ( $H=4$ ) improved the trailing-window Spectral Efficiency (SE) by approximately 6.9% for the DDQN/SEQ-DDQN pair and 1.7% for the D3QN/SEQ-D3QN pair compared to their single-step counterparts.
Baseline Comparison: The Geometric Heuristic achieved the highest final and trailing-window SE among all methods in the deterministic setting. However, the learning controllers achieved higher regularized rewards due to lower phase-switching penalties.
Sequence Length Sensitivity: The relationship between history length and performance was non-monotone. While $H=4$ provided gains, increasing the history to $H=8$ or $H=16$ did not consistently improve SE and increased input dimension and storage costs linearly.
Robustness: In the auxiliary Rician channel extension (5 distinct fading streams), both stacked-input variants yielded positive SE differences in four of the five streams, though the magnitude of improvement varied with user count and IRS size.

Significance and Claims
The paper claims that short-history state stacking serves as a compact and effective baseline for enhancing feed-forward DRL controllers in limited-observation IRS control. The results demonstrate that:

Explicitly stacking recent observations can strengthen temporal input representation without the complexity of recurrent neural networks (RNNs) or attention mechanisms.
The performance benefit of stacking is measurable but config-dependent; it is not a universal improvement across all user counts, IRS sizes, or sequence lengths.
The proposed $H=4$ configuration represents a "compact operating point" that balances performance gains against the increased dimensionality and storage costs of the replay buffer.

The authors emphasize that the Geometric Heuristic serves as a structured, position-aware benchmark rather than a theoretical upper bound, and that the observed gains highlight the value of temporal context in feedback-driven control when instantaneous CSI is unavailable. Future work suggested includes isolating specific history components and comparing explicit stacking against learned temporal encoders under richer channel models.

More like this