Technical Summary: A Data-Free Symbolic Regression Approach for Solving Equations

Problem Statement

Many equations in science and engineering, particularly those involving transcendental terms or complex differential operators, cannot be solved by existing analytical techniques. Consequently, they are typically addressed using numerical methods that yield approximate solutions without explicit symbolic expressions. While numerical solutions are quantitatively accurate, they often obscure the underlying functional dependencies, symmetries, and qualitative behaviors of the system.

Existing approaches to recovering explicit solutions face a trade-off:

1. Analytical Solvers: Can derive exact symbolic solutions but are limited to specific equation structures (e.g., polynomial systems or restricted classes of ODEs).
2. Symbolic Regression: Can discover interpretable expressions but fundamentally relies on supervised paired input-output data, which is often unavailable when only the governing equation and boundary conditions are known.
3. Residual-Based Neural Methods (e.g., PINNs): Operate directly from governing equations and constraints without needing labeled data. However, they produce solutions encoded in neural network parameters (numerical approximations) rather than explicit symbolic forms, limiting further analytical utility.

There is a gap in the literature for a method that operates directly from governing equations and auxiliary constraints, avoids the need for supervised input-output data, and recovers the solution in an explicit symbolic form.

Methodology: Symbolic Equation Solver (SES)

The authors propose the Symbolic Equation Solver (SES), a framework that formulates equation solving as an optimization problem over differentiable symbolic models. Unlike traditional symbolic regression, SES does not fit a model to data; instead, it optimizes a symbolic expression to satisfy the governing equation and its constraints.

Core Architecture

• Symbolic Model Class: SES utilizes a differentiable symbolic network based on the Equation Learner (EQL) formulation. In this architecture, conventional neural activation functions are replaced by a library of symbolic operators (e.g., identity, constants, squaring, exponential, hyperbolic tangent, multiplication).
• Differentiability: The network is fully differentiable, allowing gradients to propagate through symbolic operators. This enables the use of standard first-order optimization methods (Adam) to optimize the network weights, which correspond to the coefficients of the symbolic expression.
• Objective Function: The training signal is derived entirely from the equation residuals. Given a governing equation $F_k(x, u, \nabla u, \dots) = 0$ and auxiliary conditions $B_j(x, u, \nabla u, \dots) = 0$, SES minimizes a loss function composed of the squared residuals evaluated at sampled collocation points:
  $$L(\theta) = \frac{1}{N_F}\sum |F_k|^2 + \frac{1}{N_B}\sum |B_j|^2$$
  For non-differential equations, an auxiliary variable is introduced to generate collocation points, treating the solution as a constant function of this variable.

Optimization Strategy

The training process involves three distinct phases to ensure convergence to a sparse, interpretable solution:

1. Residual Minimization: The model is trained for 50,000 epochs using only the residual objective to satisfy the equation.
2. Sparsity Induction: Training continues for 200,000 epochs with an added $\ell_1$-penalty on internal weights. During this phase, a pruning mechanism removes 25% of the weights with the lowest contribution to the loss periodically, encouraging a compact symbolic structure.
3. Fine-Tuning: The $\ell_1$-penalty is removed, and the remaining weights are fine-tuned for 20,000 epochs using only the residual objective.
4. Extraction: The final symbolic expression is extracted by traversing the pruned computational graph and rounding the learned weights to five decimal places.

Key Contributions

1. Data-Free Symbolic Recovery: SES demonstrates that explicit symbolic solutions can be recovered without any paired input-output training data, relying solely on the mathematical structure of the governing equations and constraints.
2. Unified Framework: The approach unifies the equation-driven optimization principle of residual-based neural solvers (like PINNs) with the interpretability of symbolic regression. It applies to both algebraic and differential equations within a single formulation.
3. Joint Optimization for Systems: The framework naturally extends to systems with multiple unknown quantities, optimizing them jointly under a shared objective function.

Experimental Results

The authors evaluated SES on a diverse set of problems, comparing the recovered symbolic expressions against known analytical solutions:

• Linear Systems: Successfully recovered exact constant solutions ($x=2, y=1$) for a system of linear equations.
• Transcendental Equations: Recovered the solution $x=1$ for the equation $e^x + x^3 = e + 1$, a problem typically requiring numerical methods as standard symbolic isolation is impossible.
• Ordinary Differential Equations (ODEs):
  • Recovered $y(t) = \tanh(t)$ for the nonlinear ODE $dy/dt = 1 - y^2$.
  • Recovered the quadratic transport solution $u(x,t) = (x-t)^2$ for a linear transport equation, preserving the correct functional form despite approximate coefficient values.
• Partial Differential Equations (PDEs):
  • Successfully solved a 2D Poisson equation with mixed boundary conditions, recovering the exact polynomial structure $u(x,y) = x^2 + y^2 + xy^2$.
  • Recovered the solution for a second set of boundary conditions for the same PDE, demonstrating robustness to different constraint configurations.

In all cases, SES produced compact symbolic expressions that matched the analytical solutions, often revealing the correct functional dependencies and symmetries.

Significance and Claims

The paper posits that equation solving can be reframed not merely as analytical derivation or numerical approximation, but as optimization in symbolic function space. The significance of SES lies in its ability to bridge the gap between equation-based formulations and symbolic recovery.

• Interpretability and Discovery: By recovering solutions in explicit symbolic form, SES exposes structural dependencies and functional relationships that are difficult to infer from purely numerical approximations. The authors suggest this capability could serve as a mechanism for scientific hypothesis generation and discovery, providing candidate analytical forms and invariances.
• Scientific Machine Learning: SES represents a step toward integrating symbolic reasoning more directly into scientific machine learning, offering a path to models that are both predictive and structurally grounded.
• Limitations and Future Work: The authors modestly note that current results are proof-of-concept on problems with known analytical solutions and relatively concise structures. They acknowledge that extending the framework to more challenging PDEs, high-dimensional systems, and chaotic dynamics will require more expressive architectures, richer operator libraries, and more robust optimization strategies.