Disaggregating Time-Series with Many Indicators: An Overview of the DisaggregateTS Package

1. Problem Statement

The paper addresses the challenge of temporal disaggregation (interpolating low-frequency data into high-frequency data) in the context of high-dimensional settings.

• The Context: Governmental and administrative agencies often release precise but low-frequency data (e.g., quarterly GDP, annual emissions). Conversely, high-frequency indicators (e.g., monthly surveys, administrative records) are abundant but often less precise or timely.
• The Goal: To estimate a high-frequency series ($y_m$) that tracks short-term dynamics while remaining temporally consistent with the observed low-frequency benchmarks ($y_q$).
• The Challenge: Traditional regression-based methods (e.g., Chow-Lin) fail when the number of high-frequency indicators ($d$) exceeds or equals the number of low-frequency observations ($n$). In these "large $p$, small $n$" scenarios, the design matrix becomes rank-deficient, making standard Generalized Least Squares (GLS) estimation impossible due to non-invertibility and high variance.

2. Methodology

The paper outlines a progression from classical methods to modern sparse estimation techniques, implemented within the DisaggregateTS R package.

A. Classical Regression-Based Techniques

The foundation is the Chow-Lin (1971) method, which assumes the high-frequency series follows a linear regression with autocorrelated errors:
$$y_m = X_m\beta + u_m, \quad u_m \sim N(0, V_m)$$

• Aggregation: A matrix $C$ aggregates the high-frequency series to match the low-frequency benchmarks (e.g., summing monthly values to get quarterly totals for flow data).
• Estimation: The coefficients $\beta$ are estimated using GLS, assuming the error term follows an AR(1) process ($u_t = \rho u_{t-1} + \epsilon_t$).
• Limitation: This requires $d < n$. If $d \geq n$, the matrix $X_q^\top V_q^{-1} X_q$ is singular.

B. Extension to High-Dimensional Settings (Sparse Temporal Disaggregation)

To handle cases where $d \gg n$, the authors adapt methods from high-dimensional statistics (Mosley et al., 2022):

1. Regularization (LASSO):
   The GLS cost function is modified by adding an $\ell_1$ penalty (LASSO) to induce sparsity:
   $$\hat{\beta}(\lambda_n | \rho) = \arg \min_{\beta} \left\{ \|V_q^{-1/2}(y_q - X_q\beta)\|_2^2 + \lambda_n \|\beta\|_1 \right\}$$
   This penalizes coefficients, effectively selecting relevant indicators and shrinking irrelevant ones to zero.

2. Debiasing:
   LASSO estimators are known to be biased. To correct this, the authors employ a refitting procedure:

   • Identify the non-zero coefficients from the LASSO solution.
   • Construct a reduced design matrix containing only these selected variables.
   • Perform a standard Least Squares estimation on this reduced subset to obtain unbiased estimates.

3. Adaptive LASSO:
   To address issues where the "Irrepresentability Condition" is violated (common in highly correlated indicators), the Adaptive LASSO is used. This weights the penalty term by the inverse of an initial estimator ($\hat{\beta}_{init}$), improving variable selection consistency:
   $$\text{Penalty} = \lambda_n \sum_{j=1}^d \frac{|\beta_j|}{|\hat{\beta}_{init,j}|}$$

4. Hyperparameter Selection:

   • $\rho$ (Autocorrelation): Estimated via profile-likelihood maximization (grid search).
   • $\lambda_n$ (Regularization strength): Selected using the Bayesian Information Criterion (BIC) rather than cross-validation, due to the small sample size of low-frequency observations.

3. Key Contributions: The DisaggregateTS Package

The primary contribution is the DisaggregateTS R package, which operationalizes these methods.

• Core Function: disaggregate(Y, X, aggMat, aggRatio, method, ...)
  • Inputs: Low-frequency target ($Y$), high-frequency indicators ($X$), aggregation matrix type (e.g., "sum" for flow data, "first" for stock data), and the ratio between frequencies.
  • Methods Supported:
    • Classical: "Chow-lin", "Denton", "Fernandez", "Litterman".
    • High-Dimensional: "spTD" (Sparse Temporal Disaggregation) and "adaptive-spTD".
• Simulation Tool: TempDisaggDGP() generates synthetic data with user-defined sparsity, autocorrelation, and aggregation ratios to test the methods.
• Flexibility: Handles both flow data (sums) and stock data (point-in-time) via the aggMat argument.

4. Results and Demonstrations

The paper validates the methodology through simulations and an empirical application.

A. Simulation Studies

• Classical Setting ($d < n$): The package successfully replicates standard Chow-Lin results, showing that temporal disaggregation captures fluctuations better than simple linear interpolation.
• High-Dimensional Setting ($d > n$):
  • Standard methods fail (cannot compute).
  • The adaptive-spTD method successfully recovers the underlying signal by selecting a sparse subset of indicators.
  • Visual Evidence: Figures 3 and 4 demonstrate that while linear interpolation produces overly smooth, unrealistic lines, the sparse temporal disaggregation method reconstructs the true volatility and dynamics of the high-frequency series.

B. Empirical Application: Carbon Intensity

• Objective: Disaggregate annual CO2 emissions for IBM (17 observations) into quarterly data using 112 high-frequency financial indicators (balance sheet, income statement, cash flow).
• Preprocessing: Variables with pairwise correlation > 0.99 were removed.
• Outcome: The adaptive LASSO selected only two significant predictors:
  1. 12-month trailing sales.
  2. Total company capital.
• Performance: The resulting quarterly emission estimates showed realistic fluctuations correlated with economic activity, whereas linear interpolation failed to capture these dynamics. This allows for the calculation of timely Carbon Intensity (emissions per unit of sales).

5. Significance

• Bridging the Data Gap: The package enables researchers to utilize the "Big Data" era's abundance of administrative indicators to improve the timeliness of low-frequency economic and environmental statistics.
• Methodological Advancement: It extends the classic Chow-Lin framework to high-dimensional spaces where $d \geq n$, a common scenario in modern econometrics that previously required ad-hoc dimensionality reduction.
• Climate Finance Application: By enabling high-frequency estimation of emissions, the tool supports more responsive climate policy, carbon accounting, and ESG (Environmental, Social, and Governance) analysis, addressing the lag inherent in annual reporting.
• Reproducibility: By providing a dedicated R package, the authors lower the barrier to entry for applying complex sparse regression techniques to temporal distribution problems.