Technical Summary: Robustly Invertible Nonlinear Dynamics and the BiLipREN

1. Problem Formulation

The paper addresses the challenge of defining and constructing robustly invertible nonlinear dynamical systems. While system inversion is fundamental to control theory (e.g., feedforward compensation, internal model control) and machine learning (e.g., normalizing flows for generative modeling), standard invertibility definitions often fail to guarantee robustness against signal perturbations and initial state mismatches.

Existing approaches to nonlinear inversion often rely on zero-dynamics analysis or normal forms, which can be difficult to construct and do not directly quantify the incremental input-output gains of the inverse map. Furthermore, while linear systems allow for minimum-phase/all-pass (inner-outer) factorizations, extending these concepts to nonlinear systems with guaranteed stability and bounded gains remains an open problem. The authors aim to:

1. Define a notion of robust invertibility where both forward and inverse systems are contracting and bi-Lipschitz.
2. Construct a differentiable neural network parameterization that guarantees these properties by design.
3. Demonstrate the utility of this framework in internal model control (IMC), surrogate loss learning for trajectory optimization, and generative trajectory modeling.

2. Methodology

A. Definition of Robust Invertibility

The authors define a system as robustly invertible if it possesses a causal inverse such that both the forward and inverse systems are:

• Contracting: State trajectories converge exponentially to a unique trajectory for a given input, regardless of initial conditions.
• Bi-Lipschitz: Both the forward map and its inverse have bounded incremental input-output gains.

This implies that bounded disturbances in the output (or initial state mismatch) result in bounded errors in the reconstructed input, and vice versa. The paper establishes that if a system is $\nu$-Lipschitz and contracting, and its inverse is $1/\mu$-Lipschitz and contracting, the system is robustly invertible with explicit bounds on the error propagation.

B. Construction via Layer Composition

To construct such systems, the paper proposes a compositional approach using two types of layers:

1. Strongly Input-Output (I/O) Monotone Dynamic Layers: These are Recurrent Equilibrium Networks (RENs) satisfying a specific dissipation inequality. A system is $(\sigma, \eta)$-strongly I/O monotone if $2\langle \Delta y, \Delta u \rangle \ge \sigma \|\Delta u\|^2 + \eta \|\Delta y\|^2$. This property ensures the system is bi-Lipschitz and invertible, with the inverse also being strongly I/O monotone.
2. Static and Dynamic Orthogonal Layers:
   • Static Orthogonal Layers: Affine maps with orthogonal weight matrices ($P^\top P = I$). These are bi-Lipschitz with gain 1.
   • Dynamic Orthogonal Layers: Linear time-invariant systems with orthogonal state-space matrices, acting as all-pass (inner) filters. While their causal inverse is unstable, they are stable in reverse time, allowing for non-causal inversion over finite batches.

By composing these layers, the authors create a Nonlinear Inner-Outer Factorization: $P = O(z) \circ G$, where $O(z)$ is a dynamic orthogonal (all-pass) layer and $G$ is a robustly invertible (minimum-phase) system.

C. The BiLipREN Architecture

The core contribution is the Bi-Lipschitz Recurrent Equilibrium Network (BiLipREN).

• Structure: A deep composition of static orthogonal layers and strongly I/O monotone REN layers: $G = O_K \circ M_K \circ \dots \circ O_1 \circ M_1 \circ O_0$.
• Parameterization: The model uses a differentiable parameterization (based on Cayley transforms for orthogonal matrices and supply-rate matrices for monotone layers) that guarantees the robust invertibility constraints hold for any learned parameters.
• Analytical Inverse: The inverse of a BiLipREN is also a BiLipREN, computable analytically by reversing the layer order and inverting each component. This avoids the need for iterative solvers during inference for the inverse model.

3. Key Contributions

1. Theoretical Framework: Introduced a definition of robust invertibility based on contraction and bi-Lipschitzness, providing tractable sufficient conditions for nonlinear state-space models.
2. Model Class (BiLipREN): Proposed a new class of recurrent neural networks that are robustly invertible by design. The model admits an analytical inverse that shares the same structural properties.
3. Nonlinear Factorization: Demonstrated that the composition of dynamic orthogonal layers and robustly invertible systems yields a nonlinear inner-outer factorization, generalizing classical linear spectral factorization.
4. Applications:
   • Internal Model Control (IMC): Applied the factorization to design IMC controllers for nonlinear systems with time delays, ensuring closed-loop contraction and zero steady-state error.
   • Surrogate Loss Learning: Used BiLipRENs to model trajectory optimization costs. The bi-Lipschitz property ensures the surrogate loss satisfies the Polyak-Łojasiewicz (PL) condition, guaranteeing global linear convergence of gradient descent and enabling direct computation of the global minimum via the inverse model.
   • Generative Modeling: Utilized the model as a normalizing flow for infinite-dimensional signals, mapping Gaussian white noise to complex trajectory distributions.

4. Experimental Results

The paper validates the framework through three illustrative examples:

• Robust Inversion (Mass-Spring-Damper):

  • A BiLipREN was trained to model a nonlinear 4-mass spring-damper system.
  • Results showed that while BiLipREN imposed stronger structural constraints than a standard Contracting REN (C-REN), it achieved comparable forward modeling accuracy.
  • Crucially, the BiLipREN provided an explicit inverse model without additional training. The inverse remained robust to adversarial attacks, whereas relaxing the Lipschitz bounds ($\mu$) improved forward accuracy but degraded inverse robustness, highlighting a trade-off.

• Nonlinear Inner-Outer Factorization:

  • The model successfully learned to factorize a stable nonlinear delay system ($y_{t+1} = 0.9 \tanh(y_t) + u_{t-3}$).
  • The learned dynamic orthogonal layer accurately approximated the 3-step time delay (all-pass component), while the outer component captured the minimum-phase dynamics.

• Internal Model Control (Quadruple Tank System):

  • An IMC controller was designed for a time-delayed quadruple tank system using the learned factorization.
  • The controller achieved zero steady-state error for step references and disturbances.
  • Experiments showed that while smaller $\mu$ values improved model fitting, they increased sensitivity to noise in the control input. A moderate $\mu$ provided the best balance between tracking performance and smooth control actions.

• Surrogate Loss Learning (Obstacle Avoidance):

  • In a 2D obstacle avoidance problem, a BiLipREN was used to learn a surrogate cost function from limited data.
  • Unlike LSTM or C-REN surrogates, which suffered from spurious local minima and sensitivity to initialization during gradient descent, the BiLipREN surrogate allowed for the direct computation of the global minimum via its analytical inverse.
  • The resulting trajectory was smooth, avoided obstacles, and outperformed the best sample in the training dataset.

• Generative Modeling:

  • The model was trained to generate trajectories for the obstacle avoidance problem.
  • Latent variables mapped back through the inverse model exhibited negligible autocorrelation and matched a standard Gaussian distribution (verified via Q-Q plots), confirming the model successfully learned the data distribution.

5. Significance and Claims

The paper claims that the BiLipREN framework bridges the gap between classical control theory concepts (minimum-phase systems, inner-outer factorization, robust control) and modern deep learning (normalizing flows, equilibrium networks).

• Robustness: By enforcing bi-Lipschitzness and contraction, the model guarantees that both prediction and reconstruction are robust to perturbations, a property often lacking in standard neural network inverses.
• Optimization Guarantees: In the context of surrogate loss learning, the framework provides theoretical guarantees (PL condition) for global convergence, addressing the issue of non-convexity in black-box optimization.
• Generality: The approach unifies finite-dimensional vector mappings (normalizing flows) and infinite-dimensional signal processing (spectral factorization) under a single dynamical systems framework.
• Practical Utility: The ability to analytically invert complex, nonlinear, recurrent models without iterative solvers opens new possibilities for real-time control and efficient trajectory optimization.

The authors conclude that while the structural constraints of BiLipREN are strong, they do not preclude high-performance modeling, as evidenced by the competitive results in the experimental sections. The framework offers a principled way to integrate stability and invertibility constraints directly into the neural network architecture.