Beyond Equidistant Assumptions: An Autoregressive Ordered Stereotype Model for Ordinal Time Series

Technical Summary: Beyond Equidistant Assumptions: An Autoregressive Ordered Stereotype Model for Ordinal Time Series

Problem Statement
Ordinal time series data, characterized by observations taking values in a finite set of ordered categories, arise frequently in fields such as healthcare, environmental science, and sports. While various modeling approaches exist—including Markov models, regression models, and discrete ARMA variants—many rely on restrictive assumptions. Specifically, standard ordinal regression models often assume that the distance between adjacent categories is equidistant or that covariate effects are proportional across all outcome thresholds (as in the proportional odds model). The authors argue that these assumptions are frequently unrealistic, particularly in applications where the interpretation of the "distance" between states is meaningful (e.g., the transition from "awake" to "quiet sleep" may represent a different magnitude of change than transitions between intermediate sleep stages). Existing flexible alternatives, such as the partial proportional odds model, often incur a substantial penalty in the number of parameters. The paper addresses the need for a parsimonious yet flexible framework that captures serial dependence without imposing equidistant category spacing.

Methodology
The paper proposes the Autoregressive Ordered Stereotype Model (AR-OSM), an extension of the Ordered Stereotype Model (OSM) originally introduced by Anderson (1984) for cross-sectional data.

• Model Formulation: The AR-OSM incorporates lagged values of the response variable ($Y_{t-1}, \dots, Y_{t-p}$) as covariates within the systematic component of the regression. For an ordinal time series $Y_t$ with $q$ categories, the log-odds of being in category $k$ versus the baseline category (1) are modeled as:
  $$\log \left( \frac{P(Y_t = k | \mathcal{F}_{t-1})}{P(Y_t = 1 | \mathcal{F}_{t-1})} \right) = \alpha_k + \phi_k \sum_{j=1}^p \beta_j Y_{t-j} + \gamma^T X_t$$
  where $\mathcal{F}_{t-1}$ represents the history of the process.
• Key Feature (Score Parameters): The model utilizes a set of monotone "score" parameters $\{\phi_1, \dots, \phi_q\}$ (constrained such that $0 = \phi_1 \le \dots \le \phi_q = 1$). Unlike standard models that assume fixed, equidistant spacing, these parameters allow the data to determine the relative spacing between categories. The regression coefficients $\beta_j$ are common across categories but are scaled by the category-specific scores $\phi_k$.
• Estimation: Parameter estimation is performed via Maximum Likelihood Estimation (MLE). To handle the monotonicity constraints on the score parameters during optimization, the authors employ a reparameterization strategy (Fernández et al., 2016) involving logit transformations and exponential increments, converting the constrained problem into an unconstrained one.
• Model Selection and Dependence: The paper utilizes Ordinal Cohen's $\kappa$ and its partial version to measure serial dependence and select the appropriate autoregressive order $p$. This metric is adapted from Weiß (2020) to account for the ordinal nature of the data, serving as an analogue to ACF and PACF in continuous time series.

Key Contributions

1. Extension to Time Series: The primary contribution is the formalization of the OSM for ordinal time series, creating the AR-OSM framework.
2. Relaxation of Equidistance: The model explicitly avoids the assumption of equidistant categories. Instead, it infers the latent spacing between states from the data, offering a more realistic representation for processes where category transitions vary in magnitude.
3. Parsimony vs. Flexibility: The AR-OSM offers a compromise between the restrictive proportional odds model and more complex, parameter-heavy alternatives. It captures category-specific effects through the score parameters without requiring a separate coefficient vector for every covariate-category interaction.
4. Integration of Covariates: Unlike some pure Markovian approaches for ordinal series, the AR-OSM retains the structure of a regression model, allowing for the inclusion of external covariates (e.g., heart rate, temperature) alongside autoregressive terms.

Results

• Simulation Study:
  • Estimator Performance: Simulations across varying sample sizes ($T=100, 500, 1000$) demonstrated that the estimators are efficient, with variance decreasing as sample size increases. While a small bias was observed in the score parameters ($\phi$) for small sample sizes ($T=100$), this bias diminished with larger samples.
  • Serial Dependence: The study investigated how the autoregressive coefficient ($\beta$) and score parameters ($\phi$) influence serial dependence. Results indicated that the sign of $\beta$ determines the direction of dependence (positive or negative), while the magnitude controls the strength. The relationship between $\beta$ and serial dependence (measured by Ordinal Cohen's $\kappa$) was found to be non-linear.
• Real-World Application (Infant Sleep Data):
  • The model was applied to a dataset of newborn infant sleep states (Awake, Quiet Sleep, Indeterminate Sleep, Active Sleep) recorded every 30 seconds.
  • Model Selection: Analysis of the partial Ordinal Cohen's $\kappa$ suggested an autoregressive order of 1 ($p=1$). Likelihood Ratio Tests (LRT) confirmed that the AR-OSM (Model 3) provided a significantly better fit than a proportional odds adjacent-categories logit model (Model 5) and models including heart rate as a covariate.
  • Interpretation of Spacing: The estimated score parameters revealed that the latent distance between "Awake" and "Quiet Sleep" was substantially larger than the distances between intermediate sleep stages. This finding contradicts the equidistant assumption of standard models and supports the biological plausibility that transitioning from wakefulness to sleep is a more distinct shift than moving between sleep stages.
  • Forecasting: In a 10-step ahead forecasting exercise, the AR-OSM outperformed the proportional odds model, predicting the true state with higher probability at each step.

Significance and Claims
The paper claims that the AR-OSM provides a flexible and parsimonious framework for modeling ordinal time series where the assumption of equidistant categories is unrealistic. By allowing the data to determine the relative spacing of states, the model offers a more accurate representation of the underlying process, as evidenced by the sleep data application. The authors assert that the model successfully captures serial dependence while respecting the ordinal structure of the data.

The paper modestly notes that while the current formulation assumes a single set of score parameters shared across covariates, future work could explore "Partial Ordered Stereotype Models" where score parameters vary by covariate. Additionally, the authors acknowledge the computational challenges associated with time-varying parameters in this framework but identify the extension to autoregressive partial models as a promising avenue for future research. The work is presented as a methodological advancement that fills a gap between restrictive proportional models and overly complex alternatives.