=== SUMMARY ===

Technical Summary: SJEPA – Learning Elegant Latent Dynamics with Hybrid Symbolic–Neural Predictors

1. Problem Formulation

Joint-embedding predictive architectures (JEPAs) learn abstract states by predicting target embeddings from context embeddings, separating predictive semantics from pixel-level variability. However, standard JEPAs typically employ opaque neural transition models. While accurate, these models do not reveal which variables interact, how actions alter the future, or whether the learned coordinates support a concise dynamical description.

The paper addresses a specific gap: Can a JEPA learn not only predictive states but "elegant" dynamics over those states? Here, "elegant" is defined operationally as a compact, parsimonious law that remains adequate for prediction. The challenge is to find the simplest adequate governing law for an informative predictive state without collapsing the representation (erasing information to make the transition trivial) or underfitting the dynamics.

2. Methodology: SJEPA Framework

The author introduces Symbolic JEPA (SJEPA), a reconstruction-free framework that decomposes the latent transition operator into a hybrid of a symbolic governing law and a regularized neural correction.

2.1 Hybrid Predictor Architecture

Given a context embedding $Z_C$ and side information $\epsilon$ (e.g., action, time offset), the target embedding $\hat{Z}_T$ is predicted by:
$$\hat{Z}_T = F_{E,\alpha}(Z_C, \epsilon) + c_\phi(Z_C, \epsilon)$$

• Symbolic Law ($F_{E,\alpha}$): A compact expression with structure $E$ (from a grammar of arithmetic, polynomial, transcendental, or domain-specific primitives) and coefficients $\alpha$. This captures dominant, reusable dynamics.
• Neural Correction ($c_\phi$): A neural network that corrects for effects the selected symbolic grammar cannot express adequately (e.g., friction, unresolved interactions, approximation errors).

2.2 Constrained Operator Compression

The core principle is constrained operator compression. The objective is to minimize the complexity of the transition operator while ensuring the representation remains informative and non-collapsed.

The optimization problem is formulated as:
$$\min_{\theta, \bar{\theta}, E, \alpha, \phi} \Omega(E) + \lambda_c R_{corr}(\phi)$$
$$\text{subject to } L_{pred}(\theta, \bar{\theta}, E, \alpha, \phi) \le \delta_{pred}, \quad (\theta, \bar{\theta}) \in \Theta_{repr}$$

• Objective: Minimize symbolic complexity $\Omega(E)$ and correction reliance $R_{corr}(\phi)$.
• Predictive Constraint ($L_{pred} \le \delta_{pred}$): Prevents underfitting; the model must be accurate enough.
• Representation Constraint ($\Theta_{repr}$): Prevents the encoder from collapsing the state to a constant vector to trivialize the transition. This is enforced via regularizers (e.g., VICReg-style variance preservation) to ensure the state remains informative and non-collapsed.

2.3 Learning Strategies

The framework supports two modes:

1. End-to-End Alternating Learning: Jointly optimizes the encoder (representation) and the symbolic/neural dynamics. The process alternates between:
   • Dynamics Search: Fixing encoders to find the simplest symbolic law for current coordinates.
   • Space Search: Fixing the symbolic structure to update encoders so the representation supports a simpler transition.
2. Frozen Encoder Learning: Uses a pretrained encoder (e.g., ViT, DINO, I-JEPA) and only learns the symbolic law and correction, serving as a diagnostic for whether existing representations expose compact dynamics.

2.4 Bayesian Extension

The paper proposes a Bayesian formulation where uncertainty is placed over the symbolic structure, coefficients, and a Gaussian Process (GP) correction. This allows the model to retain multiple competing explanations when data is insufficient to identify a single law decisively.

3. Key Theoretical Insights

• Non-Identifiability of Predictive Coordinates: Predictive coordinates are not unique; any invertible transformation of the latent space preserves prediction accuracy. However, the symbolic complexity of the transition law varies across coordinate systems. Operator compression acts as an inductive bias to select coordinates where the transition is simplest.
• The Collapse Shortcut: Without explicit representation constraints, minimizing operator complexity creates a direct shortcut to representation collapse. The encoder can map all observations to a constant vector $z_0$, allowing an identity predictor to achieve zero error with zero complexity. Representation constraints are essential to prevent this.
• Allocation Control: The decomposition between symbolic and neural components is not identifiable by prediction loss alone. Regularization on the correction term ($R_{corr}$) is required to prevent the neural component from absorbing dynamics that the symbolic grammar could represent, ensuring the symbolic law retains the dominant mechanism.

4. Experimental Results

The author validates the framework using controlled pendulum experiments.

Experiment 1: Joint Coordinate and Equation Learning

• Setup: A pendulum system with high-dimensional, noisy observations. Compared Neural JEPA, post-hoc symbolic regression (fitting symbols to frozen Neural JEPA coordinates), and SJEPA (joint learning).
• Findings:
  • Simplicity: Joint SJEPA reduced mean symbolic complexity by a factor of ~5.6 compared to post-hoc fitting (from 26.0 to 4.67).
  • Accuracy: SJEPA achieved significantly lower physical-state rollout error and Out-of-Distribution (OOD) divergence than post-hoc symbolic regression, though the flexible Neural JEPA remained the most accurate in raw prediction.
  • Collapse Verification: An unconstrained one-step diagnostic (removing representation constraints) resulted in near-constant embeddings and a zero vector field, confirming the theoretical collapse shortcut.

Experiment 2: Hybrid Dynamics under Grammar Misspecification

• Setup: The true dynamics included quadratic drag ($p|p|$), but the symbolic grammar was intentionally restricted to exclude this term.
• Findings:
  • Regularization Effect: With correction regularization, the symbolic component retained the representable terms (e.g., $\sin(q)$), and the neural correction focused on the residual drag. The normalized correction-energy ratio was low (0.06).
  • Without Regularization: The unregularized hybrid allowed the neural correction to absorb the representable dynamics, shrinking symbolic coefficients and increasing the correction-energy ratio to 0.55.
  • Conclusion: Regularization successfully controls the allocation, preserving the symbolic mechanism for representable dynamics while using the neural component for residuals.

5. Significance and Claims

The paper claims SJEPA provides a complementary direction within the JEPA family, distinct from standard neural predictors or post-hoc symbolic fitting.

• Controllable Trade-off: SJEPA does not claim universal superiority in raw predictive accuracy over flexible neural networks. Instead, it offers a controllable trade-off among predictive fidelity, representation quality, symbolic parsimony, and symbolic-neural allocation.
• Elegant Dynamics: It demonstrates that operator compression can select predictive coordinates whose induced dynamics are simple yet adequate, provided representation constraints prevent collapse.
• Interpretability: The resulting models offer compact, inspectable governing laws (e.g., oscillator-like cross-coupling) that can be differentiated, linearized, and used for planning, unlike opaque neural predictors.
• Modularity: The framework is modular, supporting alternating learning, frozen encoders, Bayesian uncertainty, and action-conditioned control.

The author remains modest, noting that the experiments are controlled diagnostics on a specific system (pendulum) and that generalization to high-dimensional visual data, real-world scientific datasets, and complex control tasks remains future work. The primary contribution is the formalization of "learning the simplest adequate dynamics" as a constrained optimization problem balancing operator compression and representation integrity.