Generalised Linear Models Driven by Latent Processes: Asymptotic Theory and Applications

1. Problem Statement

Existing regression frameworks for time series data, particularly those driven by latent processes (e.g., Zeger, 1988; Davis et al., 2000; Davis & Wu, 2009), face several limitations:

• Restricted Distributional Assumptions: Most existing models rely on one-parameter exponential families (e.g., Poisson, Negative Binomial with fixed dispersion) or specific link functions.
• Additive Latent Structures: Previous formulations often model the latent effect additively within the link function (e.g., $h(x^\top \beta + \nu_t)$). This structure makes it difficult or impossible to model positive continuous data (like Gamma or Inverse Gaussian distributions) because the required mathematical conditions for asymptotic theory (such as $E[h(x^\top \beta + \nu_t)] = h(x^\top \beta)$) are hard to satisfy or impossible to verify for non-exponential link functions.
• Inference Issues: Standard Generalised Linear Model (GLM) estimators that ignore the latent process yield inconsistent standard errors because they fail to account for temporal dependence.
• Prediction Gap: There is a lack of principled approaches for prediction and forecasting in GLMs driven by latent processes in the existing literature.

The authors aim to introduce a flexible class of GLMs driven by latent processes that can handle count, real-valued, binary, and positive continuous time series, while providing rigorous asymptotic theory and valid inference methods.

2. Methodology

Model Specification

The authors propose a model where the time series $\{Y_t\}$, conditional on a latent stationary process $\{\nu_t\}$, follows a bi-parameter exponential family (EF):
$$Y_t | \nu_t \sim EF(\mu_t \nu_t, \phi)$$
where:

• $\mu_t = h(x_t^\top \beta)$ is the conditional mean determined by covariates $x_t$ and coefficients $\beta$.
• $\nu_t$ is a latent process with $E(\nu_t) = 1$ and finite second moments.
• Multiplicative Structure: Crucially, the latent process enters the mean multiplicatively ($\mu_t \nu_t$) rather than additively. This allows the model to accommodate distributions like Gamma and Inverse Gaussian, which are infeasible under additive latent structures.
• $\phi$ is a dispersion parameter, which can be unknown and estimated.

Latent Processes

The paper considers three types of stationary, strongly mixing latent processes:

1. Log-normal AR(1): $\nu_t = \exp(Z_t)$ where $Z_t$ is an AR(1) process.
2. Gamma AR(1) (GAR): A process with Gamma marginals and specific conditional density involving modified Bessel functions.
3. Squared ARCH(1): $\nu_t = Z_t^2$ where $Z_t$ follows an ARCH(1) process.

Estimation and Inference

• Parameter Estimation: The regression coefficients $\beta$ are estimated by maximizing the GLM pseudo-log-likelihood, ignoring the latent process (treating it as part of the error structure). The dispersion parameter $\phi$ and latent process parameters are estimated via the Method of Moments (MM).
• Asymptotic Theory: The authors establish the asymptotic normality of the GLM estimator $\hat{\beta}$. They utilize the Central Limit Theorem for strongly mixing processes (Peligrad & Utev, 1997).
• Correct Information Matrix: A key theoretical contribution is the derivation of the correct asymptotic variance matrix (information matrix) for $\hat{\beta}$. The standard GLM variance estimator is shown to be incorrect because it ignores the latent dependence. The correct variance involves two components:
  1. $\Omega_I$: The standard GLM information matrix.
  2. $\Omega_{II}$: A term capturing the autocovariance of the latent process.
     The asymptotic distribution is given by:
     $$\sqrt{n}(\hat{\beta} - \beta_0) \xrightarrow{d} N\left(0, \Omega_I^{-1} (\phi \Omega_I^\dagger + \Omega_{II}) \Omega_I^{-1}\right)$$

Prediction and Forecasting

The paper proposes a prediction framework based on the conditional expectation $E(Y_{t+h} | Y_t)$. Since the models are generally non-Markovian, the authors derive explicit expressions for the conditional expectation $E(\nu_{t+h} | \nu_t)$ for the specific latent processes:

• Log-normal AR(1): Closed-form solution exists.
• Gamma AR(1): Closed-form solution exists (involving conditional expectations of Gamma variables).
• ARCH(1): No closed form; approximated via Monte Carlo simulation.

3. Key Contributions

1. Generalized Framework: Extends latent-process GLMs from one-parameter to bi-parameter exponential families, enabling the modeling of positive continuous data (Gamma) and unknown dispersion parameters.
2. Multiplicative Latent Effect: By using a multiplicative latent effect ($\mu_t \nu_t$), the model avoids restrictive conditions on link functions required by additive models (Davis & Wu, 2009), making it applicable to a broader class of distributions.
3. Rigorous Asymptotics: Provides a proof of asymptotic normality for GLM estimators under this new formulation and derives the correct information matrix for valid standard error estimation, correcting the bias inherent in standard GLM inference.
4. Prediction Methodology: Fills a literature gap by providing a principled approach to forecasting in latent-process GLMs, including explicit formulas for common latent processes and Monte Carlo approximations for others.
5. Empirical Validation: Demonstrates that alternative latent processes (specifically Gamma AR(1)) can outperform the traditional Log-normal AR(1) in real-world applications.

4. Results and Applications

The authors validate the framework using two real-world datasets:

A. Measles Infections (North Rhine-Westphalia, Germany)

• Data: Weekly counts (646 observations).
• Model: Poisson GLM with Log-normal AR(1), Gamma AR(1), and Squared ARCH(1) latent processes.
• Findings:
  • Standard GLM inference (ignoring dependence) yields significantly smaller standard errors compared to the bootstrap and the proposed correct information matrix, leading to false positives.
  • Prediction Performance: The GLM-GAR (Gamma AR(1)) model achieved the best performance with the lowest Root Mean Square Error (RMSE = 8.724) and highest correlation (0.917) compared to GLM-LNAR and GLM-ARCH.
  • This highlights that the Gamma latent process is a superior fit for this count data compared to the standard Log-normal assumption.

B. Paleoclimatic Glacial Varves

• Data: Annual sediment thickness (634 years, positive continuous data).
• Model: Gamma GLM with inverse link, driven by Log-normal AR(1) and Gamma AR(1) latent processes.
• Findings:
  • Similar to the measles data, standard GLM inference incorrectly identified the trend as significant, whereas the corrected inference (accounting for dependence) showed the trend was not significant.
  • Prediction Performance: Both GLM-LNAR and GLM-GAR performed well, with GLM-GAR showing a slight edge (RMSE = 16.065 vs 16.098) and higher correlation (0.612 vs 0.610).
  • The ARCH model was discarded for this dataset as the method of moments estimator for the dispersion parameter yielded a negative value.

5. Significance

This paper significantly advances the field of time series regression by:

• Broadening Applicability: Allowing researchers to model positive continuous data (Gamma) within a latent-process GLM framework, which was previously theoretically difficult.
• Ensuring Valid Inference: Providing the necessary theoretical corrections to standard errors, preventing misleading conclusions in hypothesis testing when latent dependence exists.
• Enhancing Forecasting: Offering a complete toolkit for prediction in these complex models.
• Practical Utility: The empirical results suggest that the Gamma AR(1) latent process is a robust alternative to the ubiquitous Log-normal AR(1), often providing better fit and predictive accuracy for both count and continuous data.

The proposed framework offers a versatile, theoretically sound, and practically flexible basis for modeling time series with latent dynamic structures across various scientific domains.