Dual-Channel Tensor Neural Networks: Finite-Sample Theory and Conformal Structure Selection

Technical Summary: Dual-Channel Tensor Neural Networks

Problem Statement
Modern machine learning applications increasingly rely on high-dimensional structured data, such as neuroimaging, genomics, and spatiotemporal networks, which naturally exist as tensors. A central challenge in learning from such data is their heterogeneous structure: real-world tensors often exhibit a combination of global, low-dimensional multilinear patterns and localized, irregular, or sparse variations that carry critical predictive signals. Existing approaches face a fundamental trade-off:

• Low-rank models (e.g., CP, Tucker, Tensor-Train) effectively summarize global dependencies but suffer from representation bias when localized signals do not align with the assumed subspace.
• Sparse or high-dimensional nonparametric models capture localized effects but ignore the underlying tensor geometry, leading to inflated effective dimensions and reduced statistical efficiency.
  Furthermore, standard uncertainty quantification methods (e.g., bootstrap or standard conformal prediction) often treat tensor inputs as unstructured vectors, yielding overly conservative estimates that fail to exploit the data's intrinsic geometry. Finally, selecting among candidate tensor decompositions (e.g., CP vs. Tucker) typically relies on heuristic validation without finite-sample statistical validity.

Methodology
The authors propose the Dual-Channel Tensor Neural Network (DC-TNN), a framework designed to learn predictive functions over tensor inputs with heterogeneous structure.

1. Structured Feature Transformation:
   The input tensor $X$ is decomposed into a global low-rank core $C$ and a sparse refinement component $V$. The transformation $X \mapsto (C, V_J)$ maps the high-dimensional tensor to a low-dimensional core tensor $C$ (capturing dominant multilinear interactions) and a sparse vector $V_J$ (capturing localized deviations), where $J$ is a selected index set. This is treated as a structured inductive bias rather than a strict generative constraint.

2. Dual-Channel Architecture:
   The model processes $C$ and the refined local features $U$ (derived from $V_J$) through two parallel, coupled neural channels:

   • Core Channel ($H_c$): Operates on the low-dimensional core tensor to learn global multilinear dependencies.
   • Refinement Channel ($H_u$): Operates on the dense tensor representation of selected local features.
   • Coupling: The channels interact via cross-channel weight tensors ($W_{cu}, W_{uc}$) at each layer, allowing the model to adaptively combine structured global information with flexible local adjustments. The architecture is agnostic to the specific decomposition type, accommodating CP, Tucker, and Tensor-Train cores.

3. Training Procedure:
   The paper details a two-stage training procedure:

   • Stage 1: Estimate the core tensor $\hat{C}_i$ outside the network using standard estimators (e.g., HOSVD for Tucker, ALS for CP).
   • Stage 2: Jointly optimize the network weights and a sparse selection tensor $W_u$ (which extracts $V_J$ from $X$) to minimize the empirical risk plus a sparsity penalty (e.g., clipped-$\ell_1$). An end-to-end alternative is also discussed in the supplementary material.

4. Structure-Aware Conformal Inference:
   To quantify uncertainty, the authors develop a conformal inference procedure that operates within the core-refinement latent space rather than the ambient tensor space.

   • Local Calibration: For each test point, a local calibration set is constructed using $K$-nearest neighbors based on a distance metric combining the core and refinement representations.
   • ROC/AUC Bands: This procedure generates distribution-free confidence bands for ROC curves and AUC values, which are sharper and more informative than those derived from unstructured vector approaches.

5. Conformal Structure Selection:
   The paper introduces a statistically grounded procedure to select between competing tensor structures (e.g., Tucker vs. CP). By constructing conformal difference-ROC bands for the performance difference between two candidate models, the method provides a finite-sample, distribution-free test to determine if one structure is significantly better than another or if they are statistically indistinguishable.

Key Contributions

• Structured Modeling Framework: A dual-channel neural architecture that jointly models low-rank global structures and sparse local refinements, generalizing both pure low-rank and high-dimensional sparse models.
• Unified Perspective: A new function class that bridges low-rank tensor models and sparse models, offering a principled view of representation learning under structural uncertainty.
• Uncertainty Quantification: A structure-aware conformal inference method that produces sharper confidence regions for ROC and AUC metrics while maintaining distribution-free validity.
• Model Selection: The first known distribution-free procedure with finite-sample validity for selecting among candidate tensor decompositions.
• Theoretical Guarantees: Non-asymptotic risk bounds showing that the effective dimension depends on the core rank and refinement sparsity ($d_{eff} = \prod R_m + |J|$) rather than the ambient tensor size, avoiding the curse of dimensionality.

Results
The authors validate the framework through simulations and a real-data analysis of the DD protein dataset (TUDataset).

• Predictive Accuracy: In synthetic experiments with both Tucker and CP ground truths, the DC-TNN consistently outperformed baseline tensor neural networks (Low-rank TRL, Stacked-weight TNN) in terms of test accuracy and Mean Squared Error (MSE).
• Uncertainty Quantification: The conformal ROC bands achieved the nominal coverage rates (e.g., 90% confidence intervals), demonstrating reliable uncertainty estimation.
• Structure Selection: The proposed selector correctly identified the true underlying structure (Tucker or CP) in synthetic regimes. In the real-world DD dataset, the procedure statistically selected the CP-based DC-TNN over the Tucker-based variant, a decision supported by the difference-ROC bands lying below the diagonal for the null hypothesis that Tucker is superior.

Significance and Claims
The paper claims that the DC-TNN framework addresses the fundamental limitation of existing methods by balancing structural bias and local flexibility. By decomposing the input into a core and a refinement, the model achieves optimal convergence rates and reduces the effective dimensionality of the learning problem.

Crucially, the authors emphasize that their work provides finite-sample, distribution-free guarantees for both inference and model selection. Unlike previous methods that rely on asymptotic assumptions or heuristic validation, the proposed conformal procedures offer rigorous statistical validity for choosing tensor structures and quantifying uncertainty in high-dimensional settings. The authors position this as a step toward a general organizing principle for statistical learning on tensor-valued data, where core-augmented inference allows for adaptive, structure-aware modeling without committing to a single structural assumption a priori.