📊 statistics

PIEFS: Physics-Informed Eigenfunction Features with Learnable Scaling

The paper introduces PIEFS, a supervised neural representation-learning framework that constructs task-adaptive spectral features by training scalar coordinate maps with empirical Gram orthogonality and a learnable, anisotropic metric to replace fixed kernels with optimized Dirichlet-regularized coordinates.

Original authors: Varvara Nazarenko, Timur Lidzhiev, Alexander Tarakanov

Published 2026-07-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Varvara Nazarenko, Timur Lidzhiev, Alexander Tarakanov

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to recognize patterns in a messy pile of data, like sorting different types of fruit or identifying handwritten numbers. Usually, computers try to do this by looking at the raw data directly. But sometimes, the data is twisted, curved, or tangled in a way that makes it hard to see the differences.

PIEFS (Physics-Informed Eigenfunction Features with Learnable Scaling) is a new tool that helps the computer "untangle" this data and find the best way to see the patterns. Here is how it works, using simple analogies:

1. The Problem: The Rigid Map

Traditional methods for organizing data are like using a rigid, pre-drawn map. They assume the world is flat and that the rules for measuring distance are the same everywhere (like a standard ruler). If your data is actually curved or twisted (like a crumpled piece of paper), a rigid map fails to show the true relationships between points.

2. The Solution: A Smart, Stretchy Ruler

PIEFS is like giving the computer a smart, stretchy ruler that can change its shape based on what it sees.

The "Physics" Part: The method is inspired by physics, specifically how heat or energy spreads out (called "Dirichlet energy"). Imagine trying to smooth out a crumpled sheet of paper. The paper naturally wants to lie flat with the least amount of wrinkles. PIEFS uses this "smoothing" idea to organize the data.
The "Learnable" Part: Unlike the old rigid maps, PIEFS learns how to stretch and twist its ruler as it trains. It doesn't just measure distance; it learns the best way to measure distance for the specific task at hand.

3. How It Builds the Map (The "Sequential" Process)

Instead of trying to build the whole perfect map at once, PIEFS builds it one layer at a time, like stacking transparent sheets of glass:

Layer 1: It finds the most important way to separate the data (the "dominant mode").
Layer 2: It finds the next most important way, but it makes sure this new layer doesn't just repeat what the first one did. It forces the new layer to be orthogonal (at a perfect 90-degree angle) to the previous ones, like adding a new dimension to a cube.
The Teacher: While it builds these layers, a "teacher" (a simple classifier) constantly checks: "Is this new layer helping us sort the fruit correctly?" If yes, it keeps it. If not, it adjusts.

4. The Three "Settings" for the Ruler

The paper tests three different ways to configure this smart ruler:

OFF (The Standard Ruler): The computer uses a normal, unchangeable ruler. It still learns the layers, but it doesn't stretch the space.
DIAG (The Stretchy Ruler): The computer learns to stretch or shrink the ruler in specific directions (like making the X-axis longer than the Y-axis) to fit the data better.
TROTTER (The Rotating Stretchy Ruler): This is the most advanced version. The computer not only stretches the ruler but also rotates it before stretching. Imagine holding a piece of paper, twisting it, and then stretching it. This allows the computer to align its view perfectly with the hidden shapes in the data.

5. What Happened in the Experiments?

The researchers tested this on several challenges:

Twisted Shapes (Two Moons & Circles): On data shaped like crescent moons or concentric rings, PIEFS was incredibly accurate. It could see the curves that a simple straight line (used by older methods) could not.
Handwritten Digits (MNIST): On a classic test of recognizing numbers, PIEFS performed very well, beating some older "unsupervised" methods that tried to learn without looking at the correct answers.
Complex Images (CIFAR-10): When tested on complex images, it did well, though it didn't always beat the very best existing methods. This shows it's a strong tool, but not a magic bullet for every single problem.

The Bottom Line

PIEFS is a supervised method, meaning it learns by looking at the correct answers while it builds its map. It creates a compact, efficient set of features (a "spectral representation") that are smooth and organized.

Key Takeaway: It's not just finding the "natural" shape of the data; it's learning a customized, task-specific shape that makes the job of sorting or classifying data much easier. It's like training a guide who doesn't just know the terrain, but knows exactly how to navigate it to get you to your destination as fast as possible.

Limitations mentioned in the paper:

It doesn't solve a fixed mathematical equation (like a standard physics solver); it learns a custom solution for the specific data.
The "rotation" part of the ruler is currently a simplified version of what's theoretically possible, and making it more complex might be too computationally expensive.
It relies on "batch" processing, meaning the perfect mathematical rules are approximated using groups of data, which can cause small fluctuations during training.

Technical Summary: PIEFS – Physics-Informed Eigenfunction Features with Learnable Scaling

Problem Statement
Spectral methods are traditionally used to construct representations from data geometry by utilizing graph Laplacians or fixed kernels. However, these approaches face two primary bottlenecks: the computational cost of eigen-decomposition scales with dataset size, and evaluating features at new points typically requires rebuilding the graph and its spectrum. Furthermore, existing neural surrogates for operator eigenproblems often rely on fixed kernels or manually selected feature scalings, limiting their adaptability to specific tasks. The authors propose a supervised framework that moves away from solving fixed, supervision-independent eigenproblems toward learning task-adaptive, smooth coordinates.

Methodology
The core of the proposed method, Physics-Informed Eigenfunction Features with Learnable Scaling (PIEFS), is a supervised neural representation-learning framework based on a modified Dirichlet energy.

Objective Function: The method minimizes a composite loss function consisting of three terms:
1. Modified Dirichlet Energy (MDE): A regularization term that encourages smoothness. Unlike standard Dirichlet energy, it applies a learnable, data-dependent linear map $A(x)$ to the input gradients: $\int \|A(x)\nabla \phi(x)\|^2 p(x) dx$ .
2. Empirical Gram Orthogonality: A penalty term enforcing approximate orthogonality among the learned coordinate maps $\phi_1, \dots, \phi_K$ on the batch level, acting as a surrogate for population-level $L^2(p)$ orthogonality.
3. Cross-Entropy Loss: A supervised term where the coordinate maps serve as inputs to a linear classifier (logits), ensuring the features are predictive for the downstream task.
Learnable Metric $A(x)$ : The matrix $A(x)$ is factorized as $A(x) = \Lambda(x)U(x)$ , where:
- $\Lambda(x)$ is a diagonal scaling matrix controlling anisotropic scaling. To ensure volume preservation, the log-eigenvalues sum to zero ( $\det \Lambda = 1$ ).
- $U(x)$ is an orthogonal matrix parameterized as a structured product of Givens rotations. This provides a low-parameter, stable approximation of the orthogonal group $SO(d)$ without requiring expensive re-orthogonalization steps like QR decomposition.
Training Schedule: PIEFS employs a sequential block-coordinate descent strategy. Coordinate maps are trained one by one (outer loop $k=1 \dots K$ ). When training map $\phi_k$ , previous maps $\phi_{1 \dots k-1}$ are frozen. The loss weights are dynamically adjusted: early in training, orthogonality and classification losses are prioritized; as residuals shrink, the MDE penalty is emphasized to refine smoothness.
Inference: Once training is complete, inference is inexpensive, utilizing the learned scalar maps and a fixed linear readout.

Key Contributions

Formulation of PIEFS: The authors formulate a supervised spectral-style representation method that couples sequential coordinate maps, empirical Gram orthogonality, and a modified Dirichlet penalty with a learnable metric.
Metric Variants: The study investigates three specific settings for the metric $A(x)$ $A (x)$ :
- OFF: Identity matrix ( $A=I$ ), equivalent to standard Euclidean Dirichlet energy.
- DIAG: Diagonal scaling only ( $A=\Lambda$ ), allowing volume-preserving anisotropic scaling.
- TROTTER: Diagonal scaling after a structured Givens rotation ( $A=\Lambda U$ ), introducing rotational invariance learning.
Comprehensive Evaluation: The framework is evaluated across five benchmarks (Two Moons, Circles, HTRU2, MNIST, and CIFAR-10 embeddings) against Random Forests (RF), Logistic Regression (LR), PCA+LR, and the unsupervised NeuralEF baseline.

Experimental Results

Nonlinear Separability: On synthetic datasets like "Two Moons," where linear classifiers fail (88.15% accuracy), PIEFS achieves near-perfect separation (99.87%), demonstrating its ability to learn nonlinear decision boundaries.
Competitive Performance: On high-dimensional tasks (MNIST, CIFAR-10 embeddings), PIEFS variants outperform the unsupervised NeuralEF baseline significantly (e.g., 95.41% vs. 82.52% on MNIST with $K=16$ ).
Baseline Comparison: On the linearly separable HTRU2 dataset, PIEFS performs comparably to RF and LR, indicating it does not degrade performance when raw features are already informative.
Orthogonality: Empirical Gram matrices show approximate orthonormality (Frobenius deviation $\approx 0.23$ on MNIST), consistent with finite-batch estimation.
Metric Impact: While the TROTTER variant is expressive, the results suggest that for high-level features (e.g., CIFAR-10 embeddings), the additional complexity of rotation scaling does not always yield improvements over the identity or diagonal variants.

Significance and Claims
The paper positions PIEFS not as a general-purpose neural eigensolver for fixed physical operators, but as a compact supervised spectral representation method.

Task-Adaptive Features: The learned coordinates are interpreted as "task-adaptive, Dirichlet-regularized coordinates" rather than eigenfunctions of a fixed, supervision-independent operator. The inclusion of the cross-entropy term biases the spectral decomposition toward the classification task.
Efficiency: The method offers a physics-inspired smoothness bias with inexpensive inference after training, avoiding the need to rebuild graphs for new data points.
Limitations and Future Work: The authors modestly acknowledge that the current Givens rotation chain is not a full parameterization of $SO(d)$, limiting expressivity. They identify optimization stability, validation on explicit operator eigenproblems, and richer metric parameterizations (e.g., dense skew-symmetric generators) as primary directions for future research. The method is presented as a promising alternative to classical spectral methods in supervised settings, though it does not uniformly dominate strong classical baselines like Random Forests on all datasets.