BEACON: Benefit-Aware Early-Exit for Automatic Modulation Classification via Recoverability Prediction

This paper proposes BEACON, a benefit-aware early-exit framework for automatic modulation classification that utilizes a lightweight predictor to identify recoverable errors and trigger deeper inference only when an accuracy gain is expected, thereby optimizing the accuracy-computation tradeoff for resource-constrained IoT devices.

The Gist

The Big Picture: The "Smart Stoplight" for AI

Imagine you are running a very fast, high-tech factory that sorts mail. The mail represents radio signals, and the factory's job is to figure out what kind of signal it is (like BPSK, QPSK, or 16QAM). This is called Automatic Modulation Classification (AMC).

In the past, we built a massive, super-smart AI robot (a Convolutional Neural Network) to do this sorting. It's incredibly accurate, but it's also heavy, slow, and eats up a lot of electricity. If you try to put this giant robot on a small, battery-powered device like a smart sensor or a drone (an IoT device), the battery dies in minutes, and the robot is too slow to keep up with real-time traffic.

The Old Solution (Early Exit):
To save energy, engineers invented a "shortcut." They put a small, quick-check station at the beginning of the assembly line. If the robot is very confident (99% sure) about the answer at this first station, it stops there and sends the answer out. It only sends the "confused" or "unsure" mail to the deep, slow, energy-hungry part of the factory.

The Problem with the Old Way:
The old shortcut relied on a simple rule: "If I'm not confident, keep going."
But the researchers found a flaw in this logic.

• Scenario A: The robot is 90% sure, but it's actually wrong. The deep factory could fix this mistake. The old rule says "Stop!" because it's confident. Result: A wrong answer.
• Scenario B: The robot is 50% sure (very confused), but the deep factory can't fix it anyway. The old rule says "Keep going!" because it's unsure. Result: Wasted energy and time for no gain.

The old system was like a manager who only asks for help when they feel nervous, even if they are confidently wrong, and keeps working alone when they are confused but the problem is unsolvable.

---

The New Solution: BEACON

The authors propose a new system called BEACON (Benefit-Aware Early-exit for Automatic Modulation Classification via recOverability predictioN).

Think of BEACON as a Smart Manager who doesn't just ask, "Am I confident?" but instead asks, "Will sending this to the big boss actually help?"

How BEACON Works (The Analogy)

1. The Quick Check (Early Exit): The signal goes to the first station. The AI makes a guess.
2. The "Benefit" Calculator (LBAP): Before deciding to stop or continue, a tiny, super-fast calculator (called the LBAP) looks at the AI's guess. It doesn't just look at how sure the AI is; it looks at what kind of confusion the AI has.
   • Analogy: Imagine a student taking a test.
     • Old System: If the student is 90% sure the answer is "A," they stop. (Even if "A" is wrong).
     • BEACON: The teacher looks at the student's reasoning. "You think it's 'A', but you're confusing 'A' with 'B'. If you had 5 more minutes, you could fix this. Send them to the study hall."
     • Conversely: "You think it's 'C', but you're confusing 'C' with 'D', and even the study hall can't figure out the difference between these two. Stop here."
3. The Decision:
   • If the calculator says, "Yes, the deep factory can fix this mistake," the signal goes deeper.
   • If the calculator says, "No, the deep factory can't help, or the first guess was already perfect," the signal stops immediately.

Why is this a Big Deal?

The paper tested this on three different versions of the AI factory. Here is what they found:

• Saving Energy: To get the same level of accuracy, the old methods needed up to 3 times more computing power than BEACON. That's like driving a car that gets 10 miles per gallon vs. one that gets 30.
• Fixing Mistakes: BEACON is much better at spotting the specific cases where a mistake can be fixed. It sends those specific "fixable" problems to the deep factory and stops the "unfixable" ones early.
• Working in Bad Weather: Whether the radio signal is clear (High SNR) or noisy and static-filled (Low SNR), BEACON adapts. It knows when to be lazy (save battery) and when to work hard (fix errors).

The Takeaway

BEACON changes the question from "How sure are you?" to "Will working harder actually help?"

By answering this question correctly, it allows smart devices to run complex AI tasks without draining their batteries or slowing down. It's a smarter way to decide when to stop working and when to keep going, ensuring that every drop of energy is used only when it truly matters.

Technical Summary

1. Problem Statement

Context: Automatic Modulation Classification (AMC) is critical for wireless communication tasks like spectrum awareness and adaptive transmission. Deep Learning (DL), particularly Convolutional Neural Networks (CNNs), has become the standard for AMC due to its ability to extract features from raw In-phase/Quadrature (I/Q) signals.
Challenge: Deploying these CNN-based AMC models on resource-constrained IoT and edge devices is difficult due to limited computational power, strict energy budgets, and real-time latency requirements.
Limitation of Current Solutions: Early-Exit (EE) strategies allow samples to terminate inference at intermediate layers if they are "confident," saving computation. However, existing EE criteria rely on confidence-based metrics (e.g., entropy, Maximum Softmax Probability, margin).
Core Insight: The authors identify a fundamental flaw in confidence-based criteria: they primarily trigger early exits when the shallow prediction is already correct and consistent with the final prediction. They fail to distinguish between:

1. Irrecoverable Errors: Cases where deeper inference will not fix the mistake.
2. Recoverable Errors: Cases where the early prediction is wrong, but the final (deeper) prediction would be correct.
   Current methods waste computation on irrecoverable errors and miss opportunities to correct recoverable ones because they only measure uncertainty, not the potential benefit of deeper inference.

2. Methodology: The BEACON Framework

The authors propose BEACON (Benefit-Aware Early-exit for AMC via recOverability predictioN), a framework that shifts the decision logic from "prediction confidence" to "recoverability prediction."

A. Theoretical Foundation: Recoverability

The framework defines a Recoverable Error (Case C01) as an instance where:

• The Early-Exit (EE) prediction is incorrect.
• The Final-Exit (FE) prediction is correct.

BEACON aims to explicitly predict the probability of this specific case occurring. If a sample is likely to be a recoverable error, the system should continue to deeper layers; otherwise, it should exit early to save resources.

B. The Lightweight Benefit-Aware Predictor (LBAP)

To implement this without adding significant overhead, BEACON uses a compact, four-layer Fully Connected Network called LBAP.

• Input: Instead of compressing the output into a scalar (like entropy), LBAP takes the full probability distribution vector ($p_e$) from the EE branch. This preserves class-specific structural information (e.g., which classes are competing), which is crucial for determining if a specific confusion is resolvable by deeper layers.
• Architecture: A small Multi-Layer Perceptron (MLP) with two hidden layers (64 and 32 neurons) and ReLU/Dropout.
• Output: A scalar score $S_R(p_e) \in [0, 1]$ representing the estimated probability that the sample belongs to the "recoverable error" category.
• Training: LBAP is trained post-hoc (after the backbone is trained) using Binary Cross-Entropy loss. The ground truth label is binary: 1 if the EE is wrong and FE is right, 0 otherwise.
• Overhead: The computational cost of LBAP is negligible (approx. 0.0007% of the ResNet-18 backbone MACs).

C. Decision Logic

During inference:

1. The backbone processes the input to the EE branch.
2. The EE probability vector is fed into LBAP to get the recoverability score.
3. Thresholding: If $S_R(p_e) < t$ (threshold), the system assumes deeper inference offers no benefit and exits early. If $S_R(p_e) \ge t$, the sample is forwarded to the Final Exit for full inference.

3. Key Contributions

1. Empirical Analysis: The paper provides a rigorous analysis showing that entropy-based criteria fail to model the computational benefit of deeper inference. It demonstrates that high-entropy samples often contain a significant portion of recoverable errors (up to 46% in certain bins) that are missed by standard confidence thresholds.
2. Novel Criterion: Introduction of a Benefit-Aware EE Criterion that explicitly targets recoverable errors rather than just uncertainty.
3. LBAP Design: A lightweight predictor that utilizes the full probability distribution to capture class-competition patterns, enabling precise estimation of error recoverability with minimal overhead.
4. Comprehensive Evaluation: Validation across three different ResNet-18-based EE configurations and varying Signal-to-Noise Ratio (SNR) regimes.

4. Experimental Results

The framework was evaluated on the RadioML 2022 (RML22) dataset using ResNet-18 backbones with exits at different residual stages.

• Accuracy-Computation Trade-off: BEACON consistently outperforms state-of-the-art baselines (Entropy, MSP, Margin, Gini, Top-3).
  • Under the same computational budget, BEACON achieves up to 24% higher overall accuracy compared to entropy-based methods.
  • To achieve the same accuracy, baseline methods require up to 2.98× higher computational cost than BEACON.
• Recoverable Error Utilization: When the Final Exit (FE) invocation rate is fixed, BEACON forwards a significantly higher proportion of recoverable errors to the deeper layers compared to baselines. This means every unit of extra computation spent is more likely to yield an accuracy gain.
• Robustness: BEACON maintains superior performance across all SNR regimes (from -20 dB to 20 dB), proving its effectiveness in dynamic wireless environments.
• Calibration: The predicted recoverability scores from LBAP are highly calibrated (absolute gap < 1%) with actual observed recoverable error rates, allowing for intuitive threshold tuning.

5. Significance and Impact

• Paradigm Shift: BEACON moves the field of Early-Exit networks from "confidence estimation" to "benefit prediction." It addresses the specific inefficiency of current methods where computation is wasted on errors that cannot be fixed or skipped when errors can be fixed.
• IoT Viability: By drastically reducing the computational cost required to achieve high accuracy, BEACON makes high-performance AMC feasible for battery-powered, resource-constrained IoT devices.
• Generalizability: While focused on AMC, the principle of predicting "recoverability" rather than just "uncertainty" offers a new design principle for efficient inference in other resource-constrained deep learning applications.

In summary, BEACON solves the "blind spot" in current Early-Exit strategies by teaching the system to recognize when deeper inference is actually useful, leading to a highly efficient, energy-aware AMC system.