Technical Summary: Multi-relational Network Autoregression Model with Latent Group Structures

1. Problem Statement

The paper addresses the challenge of modeling and quantifying the effects of multi-relational networks on high-dimensional tensor-valued time series. In many real-world scenarios (e.g., social media reviews, international trade, financial markets), entities interact across multiple distinct network layers (e.g., social connections, spatial proximity, product similarity). Existing literature has largely focused on:

• Single-network autoregression: Modeling time series within one network structure.
• Vector Autoregression (VAR): Stacking tensor data into vectors, which destroys intrinsic multidimensional structure and leads to an explosion in the number of parameters ($O(\prod N_l^2)$), making estimation difficult and interpretation opaque.
• Tensor Factor Models: While effective for dimension reduction, these often lack direct interpretability regarding specific network effects and do not explicitly model heterogeneous group structures across multiple network dimensions.

Furthermore, nodal heterogeneity is prevalent; entities within the same network often exhibit distinct behavioral patterns. While latent group structures have been used in panel data and single-network time series, extending these to multi-relational tensor time series presents significant theoretical challenges due to the complex interactions between group structures across different network dimensions.

2. Methodology: The GTNAR Model

The authors propose the Group Tensor-valued Network Autoregression (GTNAR) model.

2.1 Model Formulation

Let $Y_t \in \mathbb{R}^{N_1 \times \dots \times N_q}$ be a tensor-valued time series observed at time $t$, indexed by $q$ distinct networks with adjacency matrices $A^{(1)}, \dots, A^{(q)}$. The model assumes that each network $l$ possesses a latent group structure with $G_l$ groups.

The model is defined as:
$$Y_{i_1 \dots i_q, t} = \sum_{l=1}^q \lambda^{(l)}_{g^{(l)}_{i_l}} \sum_{k=1}^{N_l} \frac{a^{(l)}_{i_l k}}{n_{l i_l}} Y_{i_1 \dots i_{l-1} k i_{l+1} \dots i_q, t-1} + \alpha_{g^{(1)}_{i_1} \dots g^{(q)}_{i_q}} Y_{i_1 \dots i_q, t-1} + \sum_{l=1}^q x^{(l)\top}_{i_l t} \zeta^{(l)}_{g^{(l)}_{i_l}} + \varepsilon_{i_1 \dots i_q, t}$$
Where:

• Network Main Effects: The term $\lambda^{(l)}_{g^{(l)}_{i_l}}$ represents the autoregressive coefficient for the $l$-th network. Crucially, this coefficient depends on the group membership $g^{(l)}_{i_l}$ of node $i_l$ in network $l$, allowing for heterogeneity within the network.
• Self-Momentum: The term $\alpha_{g^{(1)}_{i_1} \dots g^{(q)}_{i_q}}$ captures the self-driven effect, which depends on the joint group memberships across all $q$ networks.
• Covariate Effects: Exogenous covariates $x^{(l)}$ have coefficients $\zeta^{(l)}$ that also vary by group membership.
• Tensor Representation: The model can be compactly written using tensor algebra involving row-normalized adjacency matrices $W^{(l)}$ and diagonal matrices of group parameters.

2.2 Estimation Algorithm

The authors develop an iterative algorithm to simultaneously estimate model parameters ($\xi$) and latent group memberships ($G$):

1. Parameter Update: Given fixed group memberships, the model reduces to a linear regression problem. The parameters are estimated via least squares (minimizing the sum of squared errors).
2. Membership Update: Given fixed parameters, group memberships are updated sequentially for each network dimension. For each node, the algorithm assigns it to the group that minimizes the local squared error, conditional on the current memberships of nodes in other dimensions.
3. Initialization: To mitigate the non-convexity of the objective function, multiple initialization strategies are employed, selecting the solution with the minimum loss.

2.3 Group Number Selection

An Information Criterion (QIC) is proposed to select the number of groups ($G_1, \dots, G_q$) for each network:
$$\text{QIC}(G) = \log\{Q(\hat{\xi}(G), \hat{G}(G))\} + \lambda(G)$$
where $\lambda(G)$ is a penalty term. The authors provide theoretical conditions under which this criterion consistently selects the true group numbers.

3. Key Contributions

1. Novel Modeling Framework: Introduction of the GTNAR model, which integrates multi-relational network structures with latent group heterogeneity for tensor-valued time series. This allows for the direct quantification of network effects while accounting for nodal heterogeneity.
2. Simultaneous Estimation: Development of an iterative algorithm that jointly estimates model parameters and latent group memberships across multiple network dimensions, a task more complex than single-network group estimation.
3. Theoretical Guarantees:
   • Consistency: Proved that both group-wise parameters and group memberships can be consistently estimated even when the number of groups is over-specified (i.e., $G_l \ge G_{l,0}$).
   • Strong Consistency: Established that estimated group memberships converge to the true memberships (up to label permutation) with probability approaching 1.
   • Asymptotic Normality: Derived the asymptotic normality of the parameter estimators, enabling valid statistical inference (e.g., confidence intervals).
   • Group Number Selection: Proved that the proposed QIC consistently selects the true number of groups.
4. Model Extensions: The framework is extended to handle interactive network effects (multiplicative across layers) and group-specific error variances (Weighted Least Squares).

4. Results

4.1 Simulation Studies

The authors conducted extensive simulations with $q=2$ networks under Stochastic Block Model (SBM) and Power-Law network structures.

• Estimation Accuracy: When group numbers are correctly specified or over-specified, the Root Mean Squared Error (RMSE) decreases as sample size ($N$) and time length ($T$) increase, approaching the performance of "oracle" estimators (which know true memberships).
• Inference: The coverage probabilities (CP) of 95% confidence intervals approach the nominal level (0.95) as sample sizes grow.
• Under-specification: When the number of groups is under-specified, estimation bias remains high, and membership clustering accuracy drops significantly, confirming the necessity of correct or over-specification.
• Group Selection: The QIC successfully identified the true group numbers in the majority of simulation replicates.

4.2 Real Data Application (Yelp Dataset)

The model was applied to a Yelp review dataset covering five North American cities (2010–2018). The response variable was the log-transformed number of reviews by user $i$ in district $j$ at time $t$.

• Networks: Social network (user friendships) and Spatial network (district adjacency).
• Findings:
  • Heterogeneity: Distinct group structures were identified for users and districts in each city.
  • Spatial Effects: Spatial network effects were consistently positive, indicating that reviews in neighboring districts tend to influence each other. The magnitude of this effect varied by district group (e.g., higher in central business districts).
  • Social Effects: Social network effects varied significantly by user group. For some groups (e.g., long-term users), the effect was negative, while for others (e.g., highly connected users), it was positive.
  • Self-Momentum: Positive self-momentum effects were observed, indicating persistence in review activity.

5. Significance and Claims

The paper claims that the GTNAR model fills a critical gap in the literature by providing a rigorous statistical framework for valid inference on multiple network effects in high-dimensional tensor time series.

• Interpretability: Unlike stacking approaches that obscure network structure, GTNAR provides clear interpretations of how specific network layers and group memberships drive temporal dynamics.
• Efficiency: By leveraging group structures and network weighting matrices, the model drastically reduces the parameter space from $O(\prod N_l^2)$ to $O(\sum G_l p_l + \prod G_l)$, making estimation feasible for large networks.
• Robustness: The theoretical results demonstrate that the method is robust to over-specification of group numbers, a common practical concern where the true number of clusters is unknown.
• Applicability: The authors highlight the model's utility in diverse fields such as economics, finance, and social sciences, citing examples like supply chain linkages, international trade, and contextual recommendation systems, though the primary empirical validation is provided through the Yelp dataset.

The authors maintain a modest tone regarding future directions, suggesting potential extensions to high-dimensional covariates, multiplicative network effects, hidden factor structures, and categorical responses, while noting that obtaining minimax convergence rates remains an open theoretical question.