⚡ electrical engineering

Quantization Limitations of Leakage Suppression in Self-Calibrating Monostatic Integrated Sensing and Communication MIMO Systems

This paper investigates how quantization noise severely degrades the performance of digital precoding schemes for leakage suppression in self-calibrating monostatic ISAC MIMO systems, providing a closed-form solution to predict this impact and validating it through numerical analysis and hardware experiments.

Original authors: Jan Adler, Florian Gast, Gerhard P. Fettweis, Rafael F. Schaefer

Published 2026-06-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jan Adler, Florian Gast, Gerhard P. Fettweis, Rafael F. Schaefer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to listen to a whisper in a room where you are also shouting. This is the core challenge of a new type of technology called Integrated Sensing and Communication (ISAC). These are devices (like future 6G phones or radars) that try to do two things at once: send out signals to "see" the world (sensing) and talk to other devices (communication).

The problem is monostatic, meaning the "shouting" (transmitting) and the "listening" (receiving) happen in the exact same box, right next to each other.

The Big Problem: The "Echo" That Drowns Everything

When you shout, a tiny bit of that sound leaks directly into your own ear before it even bounces off a wall. In electronics, this is called leakage.

The Analogy: Imagine trying to hear a pin drop in a library, but you are also holding a megaphone right next to your ear. Even if you aim the megaphone away, the sound leaks into your ear so loudly that it blows out your hearing (saturates the amplifier) and makes it impossible to hear the pin drop (the actual signal you want to detect).

The Proposed Solution: Digital "Noise Canceling"

Engineers have a clever trick to fix this. Instead of building better physical walls, they use digital precoding.

The Analogy: Think of the megaphone as having many tiny speakers (antennas). The computer calculates a specific pattern of sound waves to play from each speaker. If done perfectly, these waves cancel each other out exactly at the spot where your ear is, creating a "silent zone" for the leakage, while still shouting loudly in the direction of the outside world.

In computer simulations (where everything is perfect), this works almost magically. The leakage disappears completely.

The Paper's Discovery: The "Pixelated" Reality

The authors of this paper asked: "Why does this magic trick fail when we build it in the real world?"

They discovered the culprit is Quantization Noise.

The Analogy: Imagine you are trying to draw a perfectly smooth, curved line on a piece of graph paper.
- In a simulation: You have infinite grid lines. You can draw a curve so smooth it looks perfect.
- In the real world (Quantization): You only have a coarse grid with very few squares (bits). You have to round your smooth curve to fit the nearest square. The result is a "jagged," pixelated line.
- The Result: That jaggedness introduces tiny errors. When you try to cancel out the loud shout, these tiny "jagged" errors mean you miss the cancellation by a tiny bit. That tiny bit is enough to let the loud shout leak through and drown out the whisper.

What They Did

The researchers built a mathematical formula (a closed-form solution) to predict exactly how much "leakage" would remain based on how "coarse" the digital grid is (how many bits the hardware uses).

The Math: They calculated that the more "pixels" (bits) you have, the smoother the curve, and the better the silence. But if you have too few bits, the "jaggedness" (noise) becomes the main reason the cancellation fails.
The Simulation: They ran thousands of computer tests. The results matched their math perfectly.
The Real-World Test: They built a physical testbed using radio equipment (SDRs) with antennas. They simulated different bit-rates by adding digital noise. The physical results confirmed their theory: The hardware's inability to represent numbers perfectly (quantization) is the primary reason leakage suppression isn't as good as the simulations promise.

The Takeaway

If you want to build a radar that listens to whispers while shouting, you can't just rely on the "perfect" math from a computer. You have to account for the fact that real hardware is "pixelated."

The paper concludes that by using their new formula, engineers can now predict exactly how well their system will work before they build it, simply by looking at the number of bits their hardware uses. This helps them design better systems without wasting time building prototypes that fail due to these tiny digital "jagged edges."

In short: The magic of digital silence-canceling is limited not by the math, but by the fact that real computers can't draw perfectly smooth lines; they can only draw jagged, pixelated ones.

Technical Summary: Quantization Limitations of Leakage Suppression in Self-Calibrating Monostatic Integrated Sensing and Communication MIMO Systems

Problem Statement
The integration of sensing functionalities into existing communication infrastructure, a key feature of the upcoming 6G standard, often relies on monostatic radar architectures where transmitting and receiving antennas are co-located within a single device. A primary challenge in realizing these systems with multi-antenna MIMO front-ends is the direct leakage of transmitted power into the receiving radio-frequency (RF) chains. This leakage can drive receiving amplifiers into saturation and clip analog-to-digital converters (ADCs). While digital transmit precoding is a promising solution for suppressing this leakage by steering signals into the nullspace of the self-interference channel, real-world deployments exhibit significantly degraded performance compared to theoretical simulations. This paper posits that quantization noise in digital-to-analog converters (DACs) and ADCs is a primary factor limiting the effectiveness of these leakage suppression schemes.

Methodology
The authors develop a simplified linear system model for a quantized monostatic Integrated Sensing and Communication (ISAC) MIMO architecture. The model treats quantization effects of uniform mid-tread ADCs and DACs as additive uniformly distributed noise. Key steps in the methodology include:

System Modeling: Defining the base-band signal flow from digital precoding weights through quantized DACs, the self-interference channel (comprising direct leakage and environmental clutter), and quantized ADCs. The model assumes a Ricean distribution for the direct leakage channel to adhere to non-clipping constraints.
Theoretical Derivation: Deriving a closed-form expression for the expected residual leakage power. This derivation accounts for the interplay between the precoding weights, the channel estimation error (specifically using a least-squares estimator), and the quantization noise introduced by both transmit and receive chains.
Validation: The theoretical model is validated through two distinct approaches:
- Monte Carlo Simulations: Using a Python-based simulator (extending HermesPy) to sweep parameters such as noise variance, observation count, and bit resolution.
- Hardware Experiments: Conducting measurements on a Software Defined Radio (SDR) testbed consisting of an Ettus X410 USRP driving a 3x3 MIMO array of horn antennas. To simulate lower bit resolutions, the authors digitally inject uniform noise into the high-resolution hardware streams (14-bit DACs and 12-bit ADCs) to mimic the effects of reduced quantization levels.

Key Contributions

Closed-Form Solution: The paper derives a closed-form expression (Equation 32) that predicts the expected residual leakage power in a self-calibrating monostatic system. This formula explicitly incorporates the impact of DAC and ADC quantization step sizes ( $\Delta_b$ ), channel estimation error variance, and precoding power.
Identification of Quantization Limits: The work identifies quantization noise as a fundamental bottleneck, demonstrating that even with perfect channel estimation, residual leakage cannot be suppressed below a floor determined by the quantization resolution of the hardware.
Empirical Validation: The study provides experimental evidence from a real-world SDR testbed that confirms the theoretical predictions, showing a near-perfect fit between the derived closed-form solution and measured data across varying bit resolutions.

Results

Simulation Results: Numerical simulations demonstrate that the theoretical model accurately predicts residual leakage power across different noise variances and bit resolutions. The results highlight a "diminishing return" effect where increasing receive bit resolution yields minimal improvement once transmit quantization noise becomes the dominant limiting factor.
Hardware Results: Measurements from the 3x3 MIMO testbed validate the theoretical model. The experimental data aligns closely with the derived expectations, confirming that quantization noise is a dominant factor in limiting leakage suppression performance. The results show that increasing the number of transmit bits significantly reduces residual leakage, whereas increasing receive bits has a limited impact if transmit quantization remains coarse.
Discrepancies: The authors note that the theoretical model diverges from numerical results in domains with very few observations, attributing this to the model's inability to accurately capture clipping effects caused by noise in low-data regimes.

Significance and Claims
The paper claims that its primary significance lies in providing a method to accurately predict the performance limits of digital leakage suppression algorithms before hardware implementation. By correcting the expected residual leakage power with a term dependent solely on known system parameters (quantization levels and channel estimation variance), developers can set realistic performance targets. The authors conclude that while their current model addresses quantization noise, future investigations must consider additional non-idealities such as clock jitter, phase noise, amplification nonlinearities, and frequency-selectivity to further refine system models. The work does not propose new algorithms but rather establishes a theoretical boundary for existing precoding schemes in practical, quantized environments.

The Big Problem: The "Echo" That Drowns Everything

The Proposed Solution: Digital "Noise Canceling"

The Paper's Discovery: The "Pixelated" Reality

What They Did

The Takeaway

More like this