🤖 machine learning

EML Trees Are Universal Approximators

This paper establishes that tree-structured compositions of the recently introduced EML (Exp-Minus-Log) function serve as universal approximators for functions in $W^{k, \infty}$ by mimicking polynomial representations and demonstrates the feasibility of a corresponding learning algorithm for practical optimization.

Original authors: Joe Germany, Elie Abdo, Joseph Bakarji

Published 2026-06-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Joe Germany, Elie Abdo, Joseph Bakarji

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a machine that can mimic any shape, curve, or pattern you can draw on a piece of paper. In the world of math and computer science, this is called "universal approximation." Most modern AI models (like neural networks) do this by stacking layers of simple switches that turn on or off.

This paper introduces a different kind of machine built from a special mathematical "Lego brick" called EML (which stands for Exp-Minus-Log).

Here is the breakdown of what the authors did, using simple analogies:

1. The Magic Brick: EML

Think of the EML function as a universal tool that combines two powerful forces: exponentials (which grow very fast, like a snowball rolling downhill) and logarithms (which shrink things down, like a telescope zooming out).

The authors discovered that if you take this specific tool and combine it with a few adjustable knobs (parameters), you can build any basic mathematical function you can think of—addition, multiplication, powers, and even complex curves like sine waves. It's like having a single type of Lego brick that can be snapped together to build a car, a house, or a spaceship, provided you know the right instructions.

2. The Construction: Building Trees

Instead of stacking these bricks in a flat line (like a standard neural network), the authors arrange them in a tree structure.

The Analogy: Imagine a family tree. At the bottom are the leaves (your input numbers). As you move up the branches, the EML bricks mix and match these numbers. At the very top (the trunk), you get the final result.
The Claim: The paper proves mathematically that no matter how wiggly or complicated a curve is, you can build a tree of these EML bricks that matches that curve almost perfectly.

3. The "Tricky" Part: The Logarithm Hole

There is one catch. The logarithm part of the EML brick hates zero and negative numbers. If you try to feed it a zero or a negative number, the math breaks (it's like trying to divide by zero).

The Solution: The authors proved that as long as you stay in the "safe zone" (numbers greater than zero), this tree works perfectly.
The Workaround: To handle the edge of the safe zone (like the number 0 itself), they proposed a clever trick: they shrink the entire problem slightly so it fits inside the safe zone, solve it there, and then stretch it back out. This ensures the math holds up even at the boundaries.

4. The Blueprint vs. The Reality

The paper does two main things:

The Blueprint (Theory): They wrote a rigorous mathematical proof showing that if you follow their specific instructions, you can build a tree that approximates any smooth function with a guaranteed level of accuracy. They even calculated exactly how big and deep the tree needs to be to get the job done.
The Reality Check (Experiments): They tried to teach a computer to build these trees using data (learning from examples rather than following the blueprint).
- The Result: The computer was very good at learning the shape of the curves. It could mimic the target functions with high accuracy.
- The Twist: However, the computer did not rediscover the neat, simple "symbolic" instructions (like $x^2 + 1$ ) that the theoretical blueprint used. Instead, it found messy, complex combinations of numbers that happened to work just as well. It's like a chef who can perfectly replicate a famous dish but uses a secret, complicated recipe that looks nothing like the original cookbook.

Summary

The paper establishes that EML trees are a theoretically perfect tool for approximating complex mathematical functions. They proved that these trees can mimic any smooth curve if built correctly. While the computer learning experiments showed they work well in practice, the "learned" versions are currently more like black boxes than the clean, readable formulas the theory suggests are possible.

In short: They proved you can build a perfect mathematical shape-shifter using these specific bricks, and they showed a computer can learn to use them, even if the computer's version is a bit messy compared to the clean theoretical design.

Technical Summary: EML Trees Are Universal Approximators

Problem Statement
The paper addresses the question of whether tree-structured compositions of the recently introduced Exp-Minus-Log (EML) function possess sufficient expressive power to serve as universal approximators for smooth functions. While previous work [22] demonstrated that EML compositions can exactly represent any elementary function (including trigonometric and exponential functions) via explicit symbolic construction, it remained an open theoretical question whether these structures could approximate arbitrary functions in Sobolev spaces $W^{k,\infty}$ with quantifiable error bounds. Furthermore, the paper investigates whether the generalized EML function, equipped with learnable parameters, can be effectively optimized in a continuous parameter space, distinct from the discrete symbolic recovery explored in prior literature.

Methodology
The authors develop a rigorous approximation theory for EML trees through a constructive proof strategy that combines local polynomial approximation with explicit algebraic representations.

Generalized EML Function: The study introduces a generalized EML atom, $EML_{\theta_i}(x,y) = a_i e^{b_i x + c_i} + d_i \ln(e_i y + f_i)$ , incorporating six learnable parameters per unit. This generalization allows for the direct encoding of coefficients, circumventing the need to recursively construct constants from the digit 1, which would otherwise lead to poor scaling in tree size.
Constructive Approximation Framework: The proof of the universal approximation theorem relies on a three-step process:
- Localization: The domain $(0,1]^d$ is partitioned into small cubes. Using the Bramble–Hilbert lemma, local polynomial approximants are constructed for the target function on overlapping cubes.
- Smoothing: To avoid the discontinuities inherent in patching local polynomials with indicator functions, the authors construct a smooth "approximate partition of unity" using shifted and rescaled hyperbolic tangent ( $\tanh$ ) functions.
- Explicit Construction: The authors demonstrate that the required algebraic operations (addition, subtraction, multiplication, division, exponentiation) and the $\tanh$ function itself can be exactly represented by EML trees. By composing these "blocks," they construct an EML tree that exactly represents the global smoothed polynomial approximation.
Handling Singularities: Since the natural logarithm is undefined for non-positive arguments, the theoretical results are initially established on the half-open domain $(0,1]^d$ . The authors extend these results to the closed domain $[0,1]^d$ by composing the approximants with an affine contraction mapping that pushes the domain into the interior, thereby controlling boundary effects.
Empirical Validation: The authors implement a learning algorithm for EML-type trees using gradient-based optimization (Adam followed by L-BFGS). They test the model on five one-dimensional benchmark functions, comparing real-valued and complex-valued parameterizations.

Key Contributions

Universal Approximation Theorem: The paper proves that EML trees are universal approximators for functions in the Sobolev space $W^{k,\infty}((0,1]^d)$ . The theorem provides explicit upper bounds on the size (number of atoms) and depth of the EML tree required to achieve a specific approximation error, scaling polynomially with the inverse of the error and the dimension.
Extension to Closed Domains: A corollary extends the approximation guarantees to the closed cube $[0,1]^d$ by utilizing an affine contraction to avoid the logarithmic singularity at zero.
Generalized Parametrization: The introduction of a six-parameter generalized EML function facilitates more compact representations of operators and enables direct coefficient encoding, which is crucial for the constructive proof and practical optimization.
Explicit Construction of Operators: The work provides detailed constructions for representing elementary operations (addition, multiplication, etc.) and the $\tanh$ function using EML trees, establishing the compositional building blocks necessary for the approximation theory.

Results

Theoretical: The paper establishes that for any $f \in W^{s,\infty}([0,1]^d)$ , there exists an EML tree $\tilde{p}_N^\theta$ such that the approximation error in the $W^{k,\infty}$ norm decays as $O(N^{-(s-k)})$ , where $N$ relates to the resolution of the domain partition. The size and depth of the tree are bounded by functions of the dimension $d$ , smoothness $s$ , and the target resolution.
Empirical: In experiments on one-dimensional targets (polynomials, $\tanh$ $tanh$ , $\sin$ $sin$ , Gaussian, and modulated sine), the generalized EML trees achieved sub-percent relative RMSE on most targets at depth 4 within a fixed computational budget.
- Real-valued parameterizations were sufficient for polynomial and hyperbolic tangent targets.
- Complex-valued parameterizations provided a measurable advantage for highly oscillatory targets (e.g., $\sin(3x)e^{-x^2/2}$ ), likely due to the ability to exactly represent trigonometric functions via Euler's identity.
- Interpretability Limitation: A "greedy snap" analysis revealed that the learned continuous parameters did not collapse to the discrete symbolic configurations (e.g., $\{0, \pm 1\}$ ) predicted by the constructive proof. Only a small fraction of parameters snapped to discrete values, and no atoms became fully discrete. This suggests that while the trees approximate the function values accurately, they do not necessarily recover the specific symbolic structure derived in the theoretical proof.

Significance and Claims
The paper claims to establish EML trees as a "theoretically grounded framework for function approximation." By bridging the gap between the symbolic exactness of elementary function representation and the statistical learning of continuous parameters, the work provides a rigorous justification for the expressive capacity of EML-based architectures.

The authors remain modest regarding the practical implications of their empirical results. They explicitly state that the study serves as a "sanity check" for the practical capability of EML trees rather than a full validation of the approximation theorem's convergence rates. They acknowledge that the learned models function as continuous approximators rather than symbolic reconstructions, noting that recovering the discrete symbolic structure would require additional structural regularization or inductive biases, which are left for future investigation. The work connects EML trees to the broader literature on universal approximation in neural networks, physics-informed neural networks, and symbolic regression, positioning them as a viable alternative for interpretable and expressive function approximation.