Technical Summary: Selective Test-Time Compute Scaling for CTR Prediction via Uncertainty-Triggered Feature Path Exploration

1. Problem Statement

The paper addresses a fundamental asymmetry in industrial Click-Through Rate (CTR) prediction: while training exposes models to diverse feature combinations, inference often encounters sparse or unseen combinations that lead to unreliable predictions. Existing training-phase solutions (e.g., adaptive gating, feature interaction architectures) learn a fixed selection function that is itself subject to the same data sparsity, offering no recourse for specific instances at deployment.

Current test-time optimization attempts in recommendation systems primarily focus on sequential behavior augmentation (e.g., item dropout), which is inapplicable to CTR's sparse categorical feature fields. Furthermore, directly transferring Large Language Model (LLM) test-time scaling techniques is infeasible due to the absence of token-level generation likelihoods in binary CTR classification. The core challenge is to develop a training-free, model-agnostic framework that scales inference depth proportionally to per-instance uncertainty without violating strict industrial latency constraints.

2. Methodology: UTTSI Framework

The authors propose UTTSI (Uncertainty-Triggered Test-Time Selective Inference), a framework that operates on top of any pre-trained CTR model without modifying its parameters or requiring retraining. UTTSI dynamically allocates computational resources based on an estimated uncertainty score for each instance.

The framework consists of three tightly integrated components:

A. Frequency Prior Estimation

To quantify the reliability of feature values, UTTSI maintains a probabilistic hashing structure (inspired by Count-Min Sketch) that tracks the frequency of each feature value in the training data. This provides an offline, data-level reliability signal. The counts are normalized to a saturation threshold to produce a frequency score $\tilde{s}_{freq}(f_i) \in [0, 1]$.

B. Dual-Signal Uncertainty Estimation

UTTSI estimates per-instance uncertainty $u(x)$ by combining two complementary signals:

1. Model-Internal Confidence ($s_{model}$): Derived from the prediction logit, normalized by a constant $\gamma$ (the 95th percentile of logits on a validation set). This captures aleatoric ambiguity near the decision boundary.
2. Data-Level Frequency Confidence ($s_{freq}$): An attribution-weighted average of feature frequencies. The attribution is calculated via the gradient norm of the logit with respect to each feature embedding ($\|\nabla_{e_i} \text{logit}(x)\|^2$). This ensures that the frequency of features actually driving the prediction is weighted more heavily than irrelevant features.

The final uncertainty score is computed as:
$$u(x) = 1 - [\alpha \cdot s_{model}(x) + (1 - \alpha) \cdot s_{freq}(x)]$$
The number of additional exploration paths $K(x)$ is allocated proportionally to this uncertainty: $K(x) = \lfloor K_{max} \cdot u(x) \rfloor$.

C. Feature Filtering and Path Exploration

The inference process proceeds in two stages:

1. Adaptive Feature Filtering (All Instances): Before any exploration, a composite score $w(f_i)$ combining frequency and attribution is computed for each feature. Features falling below a per-field threshold $\tau_i$ (computed offline from training data quantiles) are discarded. This removes unreliable embeddings for all instances.
2. Stochastic Path Exploration (Uncertain Instances Only): For instances where $K(x) > 0$, UTTSI generates $K(x)$ additional inference paths. Each path is constructed by stochastically sampling subsets of the refined feature set. Sampling probabilities are guided by the composite score $w(f_i)$, ensuring informative rare features are preferentially retained.
3. Consistency-Weighted Aggregation: Predictions from the base refined path and all $K(x)$ exploration paths are aggregated. A consistency-weighted voting scheme assigns higher weights to predictions aligned with the ensemble mean, down-weighting outlier paths that likely masked critical features.

Confident instances ($K(x)=0$) bypass the exploration stage entirely, incurring only the cost of the initial forward pass and filtering.

3. Key Contributions

• First Training-Free Test-Time Framework for CTR: UTTSI is presented as the first model-agnostic framework that selectively scales inference depth based on per-instance uncertainty without retraining or architectural changes.
• Dual-Signal Uncertainty Estimator: The paper introduces a novel estimator that combines model logit confidence with a frequency-based prior, effectively distinguishing between epistemic uncertainty (sparse coverage) and aleatoric ambiguity.
• Asymmetric Inference Design: The framework employs an asymmetric strategy where all instances undergo lightweight feature filtering, but only uncertain instances receive expensive multi-path exploration. This balances performance gains with computational efficiency.
• Empirical Validation: The approach is validated across four datasets (Criteo, Avazu, KDD12, and an Industrial dataset) using three backbone architectures, alongside a seven-day online A/B test.

4. Experimental Results

• Offline Performance: UTTSI consistently outperforms state-of-the-art CTR models (including DeepFM, HSTU, PLE, and OptFu) across all datasets. When applied to the OptFu backbone, it achieved AUC improvements of +0.0037 to +0.0048 and LogLoss reductions of -0.0116 to -0.0142. Gains were most pronounced on datasets with high feature sparsity (KDD12 and Industrial).
• Ablation Studies: Removing the dual-signal estimation, attribution-guided sampling, or multi-path aggregation resulted in consistent performance degradation, confirming the necessity of each component.
• Uncertainty Calibration: The estimated uncertainty score showed a strong Spearman rank correlation ($\rho=0.91$) with actual prediction error on the Industrial dataset, validating that the system correctly identifies difficult instances.
• Online A/B Test: In a seven-day deployment on a large e-commerce platform, UTTSI achieved a 5.3% relative improvement in Click-Through Rate (CTR) compared to a production baseline, with statistical significance ($p < 0.01$).
• Computational Overhead: The average computational cost is approximately 2.8× the base model cost. However, because the majority of instances (approx. 62%) are confident and bypass exploration, and because paths are parallelizable, the worst-case latency remains equivalent to a single forward pass.

5. Significance and Claims

The paper claims that UTTSI establishes selective inference at test time as a principled and complementary direction to training-phase advances. By deferring the allocation of additional computation until the actual instance is observed, the framework maximizes the return on already-trained parameters. It specifically addresses the "reliability gap" in CTR prediction caused by feature sparsity, offering a solution that is both effective in improving prediction accuracy and feasible for industrial deployment due to its manageable overhead and parallelizable nature. The authors emphasize that this approach does not compete with training-phase optimizations but rather enhances them by adapting inference behavior to the specific reliability of each input instance.