Spectral Discovery of Continuous Symmetries via Generalized Fourier Transforms

This paper proposes a novel framework for discovering continuous one-parameter symmetries by leveraging the Generalized Fourier Transform to detect structured sparsity patterns in the spectral domain, offering a principled and interpretable alternative to existing generator-based optimization methods.

The Gist

Imagine you are trying to teach a robot to recognize a specific type of dance. The dancer spins, twirls, and moves in complex ways, but no matter how they move, the core "vibe" of the dance remains the same. In the world of science and AI, this unchanging core is called symmetry.

Usually, scientists have to tell the robot exactly what the symmetry is beforehand (e.g., "It's a rotation!"). But in the real world, we often don't know the rules. We just have the data (the dance moves) and need to figure out the hidden rules ourselves.

This paper proposes a clever new way to find those hidden rules without guessing. Here is the breakdown using simple analogies:

1. The Old Way: Guessing the Engine

Imagine you have a car, but you don't know how the engine works. The old methods of finding symmetry are like trying to figure out the engine by tinkering with the parts.

• Researchers would try thousands of different "generator" settings (like turning a screw here, adjusting a valve there) to see which one makes the car run smoothly.
• It's like trying to find the right key to a lock by trying every key in a giant ring. It works, but it's slow, messy, and the result is often hard to understand.

2. The New Way: Listening to the Music (Spectral Discovery)

The authors suggest a different approach: Don't look at the engine; listen to the music.

They use something called a Generalized Fourier Transform. Think of this as a high-tech music analyzer.

• If you take a complex sound (like a symphony) and run it through this analyzer, it breaks the sound down into individual notes (frequencies).
• The paper's big "Aha!" moment is this: If a system has a hidden symmetry, its "music" will have a very specific pattern of silence.

The "Silent Notes" Analogy

Imagine a song where the drummer is only allowed to play on the beat, never on the off-beat.

• If you analyze the song, you will see a massive gap (silence) where the off-beat notes should be.
• That silence isn't random; it's a signature. It tells you exactly what the rule of the song is.

In this paper, the "song" is the data (like the movement of a double pendulum or particles in a collider). The "notes" are the mathematical frequencies. The "silence" (or sparsity) reveals the hidden symmetry.

How It Works in Practice

The researchers built a system that does three things:

1. Aligns the View: Imagine looking at a spinning top from a weird angle; it looks chaotic. The system first rotates its "camera" to find the perfect angle where the spinning looks simple (like a flat circle).
2. Breaks it into Notes: It translates the data into a frequency spectrum (like a piano roll).
3. Finds the Silence: It looks for the specific notes that are missing. Because of the hidden symmetry, certain notes must be zero.
   • Example: If the data is invariant to a specific rotation, the math shows that only certain "frequencies" can exist. All the others must be zero.

Why This is a Big Deal

• It's Transparent: Instead of a "black box" AI that just says "I think this is a rotation," this method gives you the exact mathematical formula for the rotation. It's like the AI saying, "I know the rule is a 45-degree turn because the 3rd and 7th notes of the song are silent."
• It's Efficient: The AI learns faster because it doesn't have to guess the rules; it just listens for the pattern of silence.
• It Works on Real Stuff: They tested this on a double pendulum (a chaotic swinging toy) and Top Quark tagging (identifying subatomic particles). In both cases, the AI successfully found the hidden physical laws that govern the motion, even though it wasn't told what they were.

The Takeaway

Think of this paper as teaching an AI to be a detective. Instead of interrogating the suspect (the data) by asking "Are you a rotation? Are you a reflection?", the detective listens to the background noise.

If the noise has a specific, structured silence, the detective knows exactly what the hidden rule is. It turns the messy problem of "finding symmetry" into a clean, mathematical game of "find the missing notes."

Technical Summary

1. Problem Statement

In many scientific and machine learning domains (e.g., physics simulations, molecular modeling), data exhibits continuous symmetries (invariance under continuous transformations like rotations) that are not known a priori.

• The Challenge: Existing methods for symmetry discovery typically rely on:
  1. Explicit knowledge: Assuming the symmetry group is known (e.g., standard CNNs for translation).
  2. Generator search: Optimizing over Lie algebra generators or learning transformation distributions (e.g., Augerino, LieGAN). These approaches often struggle with interpretability, convergence, or separating the discovery of symmetry from the predictive task.
• The Goal: To automatically discover unknown one-parameter subgroups of the Special Orthogonal group $SO(n)$ directly from data, while simultaneously learning a predictive model that respects these symmetries.

2. Core Methodology

The authors propose a spectral framework based on the Generalized Fourier Transform (GFT). Instead of searching for transformation generators in the input space, they analyze the spectral sparsity induced by invariance in the frequency domain.

Key Theoretical Insight

• Spectral Sparsity: If a function $f$ is invariant under a one-parameter subgroup generated by a skew-symmetric matrix $B$, its spectral decomposition (GFT) exhibits a specific structured sparsity.
• Resonance Condition: On the maximal torus of $SO(n)$ (which decomposes into independent 2D rotation planes), invariance imposes a linear constraint on the Fourier frequencies. Specifically, if $\lambda$ represents the angular velocities (rotation rates) of the subgroup and $m$ represents the frequency indices, the Fourier coefficient $\hat{F}(m)$ is non-zero only if:
  $$\langle m, \lambda \rangle = 0$$
  This means the spectral mass concentrates only on "resonant" frequencies that are orthogonal to the generator's rotation rates.

The Learning Framework

The proposed architecture jointly learns the symmetry parameters and the predictor through three main components:

1. Learnable Alignment ($Q$):

   • Since the invariant planes of the unknown subgroup are not known, the model learns an orthogonal matrix $Q \in SO(n)$ to align the input coordinates with these planes.
   • The input $x$ is transformed to $z = Q^T x$, decomposing the space into $n/2$ independent 2D blocks.

2. Spectral Feature Construction:

   • Each 2D block is converted to polar coordinates $(r_k, \theta_k)$.
   • Fourier Features: The model constructs features based on the characters of the maximal torus: $\exp(i \langle m, \theta \rangle)$.
   • Primitive Directions: To avoid redundancy, only "primitive" frequency vectors (coprime integer vectors) are retained.

3. Resonance Regularization:

   • The model learns the generator parameters $\lambda$ (rotation rates) and the predictor weights $w$.
   • A regularization term is added to the loss function to penalize Fourier modes that violate the resonance condition:
     $$\mathcal{L}_{total} = \mathcal{L}_{pred} + \mu \sum_{m \in \mathcal{M}} (C_m \langle m, \lambda \rangle)^2$$
     where $C_m$ is the weight associated with frequency $m$. This forces the network to suppress non-resonant frequencies, effectively "discovering" the $\lambda$ that satisfies the invariance.

3. Key Contributions

1. Spectral Framework for Discovery: A novel methodology that identifies continuous one-parameter subgroups by detecting structured sparsity in the GFT domain, rather than optimizing generators directly.
2. Spectral Architectures: The design of models that incorporate maximal-torus Fourier features, making the frequency structure interpretable and allowing symmetry discovery to emerge naturally during training.
3. Resonance-Based Regularization: A principled regularization mechanism derived from representation theory that enforces invariance by penalizing off-resonant modes, providing a bias toward functions consistent with the underlying symmetry.
4. Unified Learning: Unlike post-hoc methods, this approach unifies symmetry discovery and predictive learning in a single end-to-end framework, ensuring the discovered symmetry is directly relevant to the task.

4. Experimental Results

The framework was evaluated on two distinct tasks: a synthetic 6D Double Pendulum system and a real-world Top Quark Tagging classification task.

• Double Pendulum (6D):
  • Symmetry Recovery: The method achieved a Cosine Similarity of 0.9999 with the ground-truth generator, significantly outperforming the baseline Augerino (0.9233).
  • Predictive Performance: It maintained competitive Test MSE (0.00298 vs. 0.01053 for Augerino) while achieving much lower Invariance Error.
• Top Quark Tagging:
  • The method improved classification accuracy (84.87% vs. 74.9% for Augerino) and successfully recovered the rotational subgroup (Cosine Similarity 0.9999).
• Robustness & Efficiency:
  • Noise: The method remained stable under high noise levels, whereas baselines degraded significantly.
  • Sample Efficiency: It achieved near-perfect symmetry recovery with fewer samples compared to baselines, demonstrating that spectral constraints act as a strong inductive bias.

5. Significance and Impact

• Interpretability: The approach provides a transparent, analytically interpretable description of the discovered symmetry (via the generator $B$ and resonance conditions), unlike "black box" generator learning methods.
• Theoretical Grounding: It bridges the gap between representation theory and deep learning by translating abstract group invariance constraints into concrete, computable spectral sparsity patterns.
• Practical Utility: By automatically discovering latent symmetries, the method enables the construction of more efficient, generalizable, and data-efficient models in domains where physical symmetries are implicit (e.g., particle physics, fluid dynamics).
• Future Directions: While currently limited to one-parameter subgroups of $SO(n)$, the framework lays the groundwork for discovering higher-dimensional or multiple commuting subgroups by extending the spectral analysis beyond the maximal torus.

In summary, this paper presents a paradigm shift in symmetry discovery: moving from optimizing transformations to analyzing spectral structure, offering a more robust, interpretable, and effective solution for learning invariant functions in continuous domains.