Conformal Prediction for Hierarchical Data

Technical Summary: Conformal Prediction for Hierarchical Data

Problem Statement
The paper addresses the challenge of constructing valid prediction regions for multivariate data exhibiting a hierarchical structure, where certain components are linear combinations of others (e.g., regional sums aggregating to national totals). While standard conformal prediction (CP) provides finite-sample coverage guarantees for multivariate data, it often ignores the structural constraints inherent in hierarchical data. Conversely, forecast reconciliation techniques exploit these linear constraints to improve point forecast coherence but have historically struggled to extend these benefits to probabilistic forecasting with rigorous uncertainty quantification. The core problem is to combine these two post-hoc procedures to produce prediction regions that are both statistically valid (guaranteeing coverage) and efficient (minimizing region size) while respecting the hierarchical coherence constraints.

Methodology
The authors propose a framework that integrates Split Conformal Prediction (SCP) with Forecast Reconciliation via a projection step. The methodology operates under an i.i.d. assumption for the data and non-conformity scores, with specific distributional assumptions introduced for efficiency analysis.

1. Hierarchical Setting: The data $y \in \mathbb{R}^m$ satisfies $y = H y_{1:n}$, where $H$ is a known structural matrix mapping the most disaggregated levels ($n$) to the aggregated levels ($m$). Vectors satisfying $u = H u_{1:n}$ are termed "coherent."
2. Two Coverage Objectives:
   • Joint Coverage: Ensuring the true vector $y_{T+1}$ falls within a multivariate prediction set $C(x_{T+1})$ with probability $1-\alpha$.
   • Component-wise Coverage: Ensuring each individual component $y_{T+1,i}$ falls within its specific interval $C_i(x_{T+1})$ with probability $1-\alpha$. This is identified as a more natural objective for hierarchical data where components are often analyzed individually.
3. Proposed Procedure (Reconciled SCP):
   • For Joint Coverage: The authors adapt the ellipsoidal SCP approach (Johnstone & Cox, 2021; Messoudi et al., 2022). They introduce a projection matrix $P_A$ (orthogonal in the $A$-norm) onto the coherent subspace $\text{Im}(H)$. The regressor is modified to $\hat{\mu} = P_A \hat{\mu}_{base}$, and non-conformity scores are computed based on the distance between observations and these projected forecasts.
   • For Component-wise Coverage: The authors utilize signed non-conformity scores ($s_t = y_t - \hat{\mu}_t$). Unlike absolute residuals, signed residuals preserve the linear coherence property ($s_t = P s_t$ if $\hat{\mu}_t$ is projected). The procedure applies SCP component-wise to these signed scores after projecting the base forecasts onto the coherent subspace.
4. Efficiency Analysis:
   • Theoretical efficiency is analyzed under the assumption that residuals follow an elliptical distribution.
   • For a fixed weight vector $w$, the authors prove that projecting the forecasts using a specific matrix $P_w$ (derived from $H$ and $w$) reduces the expected sum of squared interval lengths compared to unprojected forecasts.
   • A stronger "oracle" result is derived assuming the covariance matrix $\Sigma$ of the scores is known. Using the Minimum-Trace (MinT) projection ($P_{\Sigma^{-1}}$), the method achieves uniform efficiency improvements over any other projection matrix.

Key Contributions

1. Novel Synthesis: This work is the first to formally combine conformal prediction and forecast reconciliation to construct valid and efficient prediction regions for hierarchical data.
2. Component-wise Coverage Guarantees: The paper introduces and analyzes a component-wise coverage criterion, which is more relevant for hierarchical applications than joint coverage. It proves that the proposed reconciled SCP (Algorithm 5) maintains these guarantees.
3. Efficiency Proofs:
   • Joint Coverage: The authors prove that the reconciled ellipsoidal prediction regions are uniformly smaller (in Lebesgue measure) than standard ellipsoidal regions, provided the projection matrix is chosen correctly.
   • Component-wise Coverage: Under elliptical distribution assumptions, the paper establishes that the reconciled prediction intervals are more efficient (smaller expected squared lengths) than standard component-wise intervals. This is achieved by relating the minimization of interval lengths to trace minimization problems central to forecast reconciliation literature (specifically, the MinT approach).
4. Technical Innovation: The authors bridge the gap between the two fields by utilizing signed non-conformity scores to handle the lack of unbiasedness assumptions typically required in forecast reconciliation, while leveraging trace inequalities to prove efficiency gains in the conformal setting.

Results
Theoretical findings are validated through extensive experiments on synthetic hierarchical data with varying complexities (up to 1,801 nodes).

• Coverage: All proposed algorithms successfully achieve the target coverage levels (90%) for both joint and component-wise objectives across all configurations.
• Efficiency (Joint): The reconciled ellipsoidal method (Algorithm 2) consistently produces smaller prediction volumes than the baseline (Algorithm 1).
• Efficiency (Component-wise):
  • The Weighted Least Squares (WLS) reconciliation strategy (using diagonal covariance estimates) consistently outperforms the baseline (Direct) and other methods (OLS, Combi), reducing total interval lengths by 15% to 65%.
  • The MinT strategy (using the full covariance matrix) performs best for small hierarchies but becomes unreliable for large hierarchies (Configurations 5 and 6) due to potential singularity issues in the covariance matrix estimation.
  • The Combi and OLS methods show mixed performance, with OLS being robust but less efficient than WLS.

Significance and Claims
The paper claims to provide a theoretical basis for improving uncertainty quantification in hierarchical settings. By demonstrating that structural constraints can be leveraged post-hoc to shrink prediction regions without sacrificing coverage, the work offers a practical alternative to computationally expensive hierarchical modeling during the training phase.

The authors are modest regarding the scope of their theoretical guarantees, noting that the analysis relies on an i.i.d. framework and elliptical distribution assumptions. They explicitly state that real-world hierarchical time series often violate the i.i.d. assumption on non-conformity scores. Consequently, the paper positions its results as a foundational step, suggesting that future work should extend these principles to non-exchangeable data (e.g., via adaptive conformal inference) and dynamic hierarchies. The primary significance lies in establishing that "reconciliation" is not just a tool for point forecasts but a valid mechanism for enhancing the efficiency of distribution-free prediction sets.