Technical Summary: Federated Learning of Nonlinear Temporal Dynamics with Graph Attention-based Cross-Client Interpretability

Problem Statement

Modern industrial systems are increasingly monitored by distributed sensors generating high-dimensional time-series data across multiple interdependent subsystems. Understanding the temporal interdependencies between these subsystems is critical for diagnostics and reliable operation. However, learning these dynamics faces four significant constraints in practical deployments:

1. Decentralization and Privacy: Raw sensor data cannot be shared due to privacy, communication bandwidth, and data sovereignty constraints.
2. Heterogeneity and Fixed Models: Subsystems often operate fixed, proprietary models (e.g., Original Equipment Manufacturer models) that cannot be modified or retrained.
3. Nonlinearity: The systems exhibit strong nonlinear dynamics, making cross-subsystem temporal interdependencies state-dependent and difficult to interpret.
4. Interpretability: Existing neural approaches often lack structured mechanisms to expose how subsystem interactions vary with the system's operating state.

The core challenge is to learn a global model of cross-client temporal interdependencies using only low-dimensional latent representations communicated from clients, without altering their proprietary internal estimators, while providing a principled interpretation of these learned dynamics.

Methodology

The authors propose a federated framework where a central server learns a graph-structured state-transition model over latent states communicated by clients.

1. Client-Side Modeling

Each client $m$ operates a fixed, proprietary Extended Kalman Filter (EKF) that estimates a local latent state $(\hat{h}^m_t)_c$ from local observations $y^m_t$. To incorporate cross-client information without modifying the proprietary EKF, the client introduces a learnable nonlinear augmentation function $\Delta_m(\cdot; \theta_m)$.

• Augmented State: The client computes an augmented state $(\hat{h}^m_t)_a = (\hat{h}^m_t)_c + \Delta_m(y^m_{t-1}; \theta_m)$.
• Local Dynamics: The augmented state evolves via its own temporal dynamics $(\tilde{h}^m_t)_a = f_m((\hat{h}^m_{t-1})_a)$, distinct from the proprietary EKF.
• Optimization: Clients minimize a local reconstruction loss and update $\theta_m$ using gradients received from the server.

2. Server-Side Modeling

The server receives the proprietary states $(\hat{h}^m_t)_c$ and augmented states $(\tilde{h}^m_t)_a$ from all clients. It models cross-client interdependencies using a Graph Attention Network (GAT).

• State Transition: The server predicts the future state of client $m$ as:
  $$(\tilde{h}^m_t)_s = \sigma\left(\sum_{n \in \mathcal{N}(m)} \alpha_{mn} W_{mn} (\hat{h}^n_{t-1})_c\right)$$
  where $\alpha_{mn}$ are attention coefficients and $W_{mn}$ are learnable transformations.
• Loss Function: The server minimizes the Mean Squared Error (MSE) between its predictions $(\tilde{h}^m_t)_s$ and the clients' augmented states $(\tilde{h}^m_t)_a$.
• Communication: The server backpropagates the loss gradient $\nabla_{(\tilde{h}^m_t)_a} L_s$ to the clients to update their augmentation parameters, creating a bidirectional coupling without sharing raw data or server states.

3. Interpretability via Jacobians

To address the nonlinearity and lack of interpretability, the paper establishes a theoretical link between the GAT attention coefficients and the Jacobian of the learned state-transition function.

• The Jacobian block $J_{mn}(t) = \frac{\partial (\tilde{h}^m_t)_s}{\partial (\hat{h}^n_{t-1})_c}$ quantifies how a perturbation in client $n$'s past state affects client $m$'s future state.
• Proposition 6.1 derives a closed-form relationship showing that $J_{mn}(t)$ depends on the attention coefficient $\alpha_{mn}(t)$, the weight matrix $W_{mn}$, and the derivative of the activation function $\sigma'$. This allows the attention coefficients to serve as a proxy for the strength of directed temporal influence, modulated by the system's current operating state.

Key Contributions

The paper claims the following technical contributions:

1. Formalization: A formalization of federated learning for cross-client temporal interdependencies in nonlinear state-space systems with fixed proprietary client models.
2. Architecture: The design of a GAT-parameterized server-side state-transition model that explicitly represents cross-client dynamical interactions over communicated latent states.
3. Interpretability: The derivation of analytical relationships between attention coefficients and Jacobian blocks, enabling principled, state-dependent interpretation of cross-client interdependencies. This is claimed to be the first such characterization in decentralized nonlinear systems.
4. Convergence Guarantees: Theoretical proof that the learned server-side dynamics and Jacobians converge to a centralized oracle (a system with access to all raw data) under specific assumptions (e.g., bi-Lipschitz activation, state independence).
5. Empirical Validation: Validation on synthetic and real-world industrial datasets demonstrating convergence, interpretability, scalability, and privacy-utility trade-offs.

Experimental Results

Synthetic Experiments

Using a controlled nonlinear state-space system with known ground-truth interdependencies:

• Structure Recovery: The server-side GAT closely matched the ground-truth attention coefficients, significantly outperforming local-only baselines.
• Jacobian Convergence: The learned Jacobian blocks converged to the centralized oracle's Jacobians, validating the theoretical bounds.
• Correlation: A strong empirical correlation was observed between attention coefficients and Jacobian magnitudes, supporting the theoretical link.
• Scalability: The framework showed robustness to increasing observation dimensions but increased loss with higher latent dimensions and client counts due to communication overhead.
• Privacy: The system remained robust to small levels of injected Gaussian noise in communications, with performance degrading only at high noise levels.

Real-World Experiments

Evaluated on the Hardware-in-the-Loop Augmented Industrial Control System (HAI) benchmark (water treatment, chemical dosing, heating processes):

• Performance: The proposed model achieved client prediction accuracy significantly better than proprietary client models and a pretrained federated baseline, approaching the performance of a centralized baseline.
• Interpretability: The time-averaged attention structure learned by the server closely matched that of the centralized baseline.
• Jacobian Agreement: High similarity was found between the Jacobians learned by the proposed server model and the centralized model.

Significance and Claims

The paper positions itself as the first work to establish interpretable cross-client temporal interdependencies in decentralized nonlinear systems. Its significance lies in bridging the gap between:

1. Privacy-preserving learning: Allowing collaboration without sharing raw data or modifying proprietary client models.
2. Nonlinear dynamics: Handling complex, state-dependent interactions that linear models cannot capture.
3. Explainability: Providing a mechanism (via Jacobian-attention analysis) to understand how and when subsystems influence one another, rather than treating the global model as a black box.

The authors note limitations, specifically that the framework assumes a known graph skeleton (adjacency matrix) and relies on local first-order approximations for interpretability, which may degrade under extreme nonlinearity. They suggest future integration with causal discovery methods to learn the graph structure.