Neural Conjugate Aggregation: Identifiable Unsupervised Multi-Sensor Regression under Heterogeneous Sensor Bias

Technical Summary: Neural Conjugate Aggregation (NCAM)

Problem Statement
The paper addresses the challenge of unsupervised multi-sensor regression, a scenario common in sensor networks, simulation ensembles, and scientific monitoring where multiple noisy, biased measurement sources are available, but ground-truth labels for the target variable are absent during training. In such settings, sensors often exhibit heterogeneous biases (multiplicative and additive), varying noise levels, and temporal drift. Classical fusion methods typically fail here because they rely on known sensor characteristics or labeled data for calibration. Furthermore, latent-variable fusion models in unsupervised settings suffer from structural non-identifiability, where multiple parameterizations can explain the same observations equally well, rendering the recovered latent state ambiguous. Existing approaches often address only subsets of these issues (e.g., supervised calibration, heuristic aggregation, or uncertainty quantification with labels) but lack a unified framework for unsupervised calibration, reliability estimation, identifiability, and uncertainty quantification simultaneously.

Methodology: Neural Conjugate Aggregation Model (NCAM)
The authors propose NCAM, a hierarchical Bayesian framework that integrates neural networks with conjugate Gaussian inference to perform unsupervised multi-source fusion.

1. Generative Model:

   • Latent Prior: A latent target variable $\Theta_i$ is modeled with a Gaussian prior $p(\Theta_i | X_i)$, where the mean and variance are outputs of a neural network conditioned on contextual covariates $X_i$.
   • Measurement Model: Each sensor $j$ provides a noisy observation $b_{ij}$ modeled as $b_{ij} = a_j(X_i)\Theta_i + c_j(X_i) + \epsilon_{ij}$. Here, neural networks parameterize sensor-specific multiplicative bias ($a_j$), additive bias ($c_j$), and source-dependent noise variance ($\sigma^2_j$).
   • Posterior Aggregation: Due to the conjugate Gaussian structure, the posterior over the latent target $\Theta_i$ given all sensor measurements is analytically tractable. The posterior mean serves as a reliability-weighted, bias-corrected fusion of all sensors, where weights are determined by the learned precision ($a_j^2 / \sigma_j^2$).

2. Resolving Non-Identifiability (Sensor Anchoring):

   • The measurement model is invariant under affine transformations of the latent variable, leading to non-identifiability. To resolve this, NCAM employs a sensor anchoring strategy: one reference sensor is fixed with multiplicative bias $a_{j^*} \equiv 1$ and additive bias $c_{j^*} \equiv 0$. This breaks the symmetry, rendering the latent state identifiable relative to the anchor and ensuring that improvements in likelihood correspond to genuine predictive accuracy.

3. Training Objective:

   • Since ground truth is unavailable, the model is trained by maximizing the marginal likelihood of the observed sensor measurements.
   • To prevent variance inflation (where noise variances grow arbitrarily without affecting likelihood), an explicit variance regularization term is added to the loss function, penalizing extreme prior and sensor variances.

4. Uncertainty Quantification:

   • Bayesian Uncertainty: The model naturally provides epistemic uncertainty (via posterior variance) and aleatoric uncertainty (via residual noise).
   • Ground-Truth-Free Calibration: To provide finite-sample coverage guarantees without labeled targets, the authors integrate Locally Adaptive Monte Carlo Conformal Prediction (MC-CP). This procedure generates synthetic realizations from the posterior predictive distribution to compute conformity scores, producing heteroscedastic prediction intervals.
   • Sensor-Anchored Calibration: A complementary mechanism evaluates the consistency of fused predictions against bias-corrected sensor observations directly, offering an external check against model misspecification.

Key Contributions

• Formulation: The paper formulates unsupervised multi-sensor regression as a probabilistic latent-state inference problem that jointly handles heterogeneous biases, reliability estimation, identifiability, and uncertainty.
• NCAM Framework: Introduction of a hierarchical Bayesian model that learns sensor bias and reliability while admitting closed-form conjugate posterior aggregation.
• Identifiability Solution: A principled sensor anchoring strategy that resolves structural non-identifiability, yielding interpretable and identifiable latent-state estimates.
• Calibration: Development of a ground-truth-free Monte Carlo conformal calibration procedure that produces locally adaptive prediction intervals without requiring labeled target observations.
• Empirical Validation: Demonstration of improved predictive accuracy and well-calibrated uncertainty on synthetic and real-world air-quality datasets compared to unsupervised baselines (mean aggregation, probabilistic PCA, Kalman filtering, and VELI).

Experimental Results
The authors evaluated NCAM on three datasets: a synthetic "Toy" dataset, SensEURCity (spatio-temporal PM2.5 from three cities), and CAIRSENSE (co-located PM2.5 sensors).

• Predictive Accuracy: NCAM achieved the best or near-best performance across all datasets. On the synthetic dataset with known heterogeneous biases, NCAM reduced RMSE by roughly 50% compared to the next best unsupervised baseline (VELI) and over 100% compared to classical methods like Kalman filtering. On real-world datasets, gains were substantial where sensor bias was significant (SensEURCity) and competitive where sensors were nearly homogeneous (CAIRSENSE).
• Identifiability & Bias: Ablation studies confirmed that without sensor anchoring, the model suffers from non-identifiability (improving likelihood while degrading predictive accuracy). Removing explicit bias modeling caused dramatic performance drops in heterogeneous settings but had negligible or slightly positive effects in highly correlated, low-bias settings.
• Uncertainty Calibration: The proposed MC-CP methods provided valid prediction intervals without ground truth. "MC-CP (Model)" yielded the tightest intervals by leveraging the model's adaptive variance, while "MC-CP Sensor-Anchored" offered a conservative, measurement-grounded alternative.

Significance and Claims
The paper claims that reliable multi-sensor regression is achievable in fully unsupervised settings by combining principled probabilistic aggregation with uncertainty calibration. The significance lies not in introducing individual components (Bayesian inference, neural networks, or conformal prediction) but in their principled integration to solve the unique, coupled challenges of unsupervised sensor fusion. The framework enables:

1. Estimation of sensor bias and reliability without labels.
2. Resolution of latent-state non-identifiability.
3. Generation of uncertainty-aware predictions with finite-sample coverage guarantees.

The authors conclude that accurate, uncertainty-aware sensor fusion is feasible without supervision, particularly in regimes with significant inter-sensor bias, while remaining competitive in homogeneous environments. Future work is suggested to explore dependence-aware calibration and extensions to multivariate targets.