Local Second-Order Adjoint Dynamics for Implicit Neural Networks
This paper introduces Causal Adjoint Transport (CAT), a local second-order adjoint dynamics method that significantly reduces the computational cost of training implicit and recurrent neural networks near stability boundaries by requiring substantially fewer Jacobian actions compared to first-order relaxation and other solvers.