⚡ electrical engineering

Majorization-Minimization Networks for Inverse Problems: An Application to EEG Imaging

This paper proposes a learned Majorization-Minimization framework for inverse problems, such as EEG imaging, that integrates a lightweight recurrent neural network to learn structured curvature majorants within a bilevel optimization setting, thereby achieving improved accuracy, stability, and generalization while preserving the theoretical convergence guarantees of classical MM methods.

Original authors: Le Minh Triet Tran (IMT Atlantique, LaTIM), Sarah Reynaud (IMT Atlantique, LaTIM), Ronan Fablet (IMT Atlantique, Lab-STICC), Adrien Merlini (IMT Atlantique, Lab-STICC), François Rousseau (IMT Atlantiq

Published 2026-05-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Le Minh Triet Tran (IMT Atlantique, LaTIM), Sarah Reynaud (IMT Atlantique, LaTIM), Ronan Fablet (IMT Atlantique, Lab-STICC), Adrien Merlini (IMT Atlantique, Lab-STICC), François Rousseau (IMT Atlantique, LaTIM), Mai Quyen Pham (IMT Atlantique, Lab-STICC)

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a giant, messy jigsaw puzzle, but you've lost half the pieces, and the ones you have are blurry and distorted. This is what scientists call an inverse problem. In the specific case of this paper, they are trying to figure out what's happening inside a human brain just by looking at electrical signals on the scalp (EEG). It's like trying to guess the shape of a hidden object inside a box just by shaking the box and listening to the noise.

Because the "noise" is so confusing and the "box" is so complex, there are millions of possible answers, and most of them are wrong. Finding the right answer is mathematically very difficult and unstable.

Here is how the authors of this paper solved it, using a mix of old-school math and new-school AI.

1. The Old Way vs. The New Way

The Old Way (Classical Math): Imagine a hiker trying to find the bottom of a valley in thick fog. They take small, careful steps downhill. This is safe, but it's incredibly slow, and they might get stuck in a tiny dip that isn't the real bottom.
The "Black Box" AI Way: Imagine hiring a super-smart robot that has seen millions of valleys. It looks at the fog and instantly guesses the bottom. It's fast, but sometimes it gets it wrong because it's just guessing based on patterns it memorized, not understanding the physics of the hill. If the fog looks slightly different than what it trained on, the robot might hallucinate a valley that doesn't exist.

2. The Authors' Solution: "The Safety-Net Hiker"

The authors created a new method called MM-Net. Think of it as a hiker who has a smart GPS (the AI) but is also tethered to a safety rope (the math).

The Smart GPS (The AI): They use a special type of AI (a Recurrent Neural Network) that learns how to take bigger, smarter steps. Instead of just walking slowly, the AI predicts the shape of the hill and suggests a shortcut.
The Safety Rope (The MM Guardrails): This is the paper's big innovation. Usually, AI is a "black box" that might take a step that looks good but actually sends you up the wrong mountain. The authors built a mathematical "fence" around the AI.
- Before the AI makes a move, the math checks: "Is this step safe? Does it guarantee we are moving closer to the solution?"
- If the AI suggests a step that is too wild or risky, the math "clips" it, forcing the AI to stay within a safe, proven path.

3. The "Cosine Similarity" Trick

In brain imaging, the size of the signal often doesn't matter as much as the direction or shape of the wave. The authors used a specific mathematical tool called Cosine Similarity to focus on the shape rather than the volume.

However, this tool is tricky to use because it's like trying to walk on a curved, slippery surface. The authors did something very clever: they mathematically calculated the "slipperiness" of this surface and built a custom safety net specifically for it. This allowed them to use the AI to speed things up without ever falling off the cliff.

4. The Results: Better, Faster, and More Reliable

They tested this "Safety-Net Hiker" on brain imaging data.

Accuracy: It found the brain activity much more precisely than the old slow methods.
Stability: Unlike the "Black Box" AI, which sometimes got confused and produced weird, fake brain activity (hallucinations), MM-Net stayed on track.
Generalization: This is the most impressive part. They trained the system on one type of brain data (simple, synthetic patterns) and tested it on a completely different type (complex, real biological patterns). The "Black Box" AI usually fails in this situation, but MM-Net worked perfectly. It learned the rules of the road (how to optimize) rather than just memorizing the scenery.

Summary

The paper introduces a system that teaches an AI how to solve complex brain puzzles. But instead of letting the AI run wild, they put it in a harness. The AI provides the speed and intelligence, while the math provides the safety guarantees. The result is a tool that is fast, accurate, and trustworthy, even when looking at brain data it has never seen before.

Technical Summary: Majorization-Minimization Networks for Inverse Problems

Problem Statement
Inverse problems, such as estimating a signal $x$ from noisy measurements $y = Lx + \epsilon$ , are typically ill-posed due to non-invertible forward operators ( $L$ ) and noise sensitivity. While learning-based approaches like deep unrolling and meta-learning have achieved strong empirical results, they often lack explicit control over descent directions and local geometry, particularly regarding the curvature of the loss function. This absence of theoretical constraints can lead to instability and poor generalization, especially in scenarios involving scale-invariant metrics like cosine similarity, where traditional first-order optimizers struggle with non-convex geometries and local minima. Furthermore, existing methods often fail to provide rigorous convergence guarantees when replacing analytical rules with learned parameters.

Methodology: MM-Net
The authors propose MM-Net, a learned Majorization-Minimization (MM) framework designed to resolve the trade-off between the acceleration of deep learning and the stability of classical optimization. The method operates within a bilevel optimization setting:

Analytical MM Guardrails: Instead of learning a full optimizer, the framework learns a structured curvature majorant. The core innovation lies in deriving explicit, state-dependent analytical curvature bounds for the cosine similarity loss. These bounds define a "safety ceiling" for the optimization steps, ensuring that every update remains within a region of monotonic descent.
Learned Adaptive Preconditioner: A lightweight Recurrent Neural Network (RNN), instantiated with ConvLSTM cells, acts as a dual-recurrent architecture to predict an adaptive diagonal inverse curvature vector. This network approximates second-order curvature information, allowing for accelerated convergence.
Projection Mechanism: Crucially, the RNN's output is not used directly. It is projected element-wise onto an analytical interval defined by the derived curvature bounds (lower bound $\nu$ for positive definiteness and upper bound $\nu_i$ based on the spectral properties of the loss). This projection ensures the learned preconditioner satisfies the rigorous MM conditions, transforming the network into a mathematically valid surrogate optimizer.
Bilevel Training: The system is trained via a supervised bilevel scheme. The lower level minimizes a variational energy composed of a cosine-similarity data fidelity term and a structural prior (encoded by an AutoEncoder). The upper level updates the network parameters to minimize a meta-loss that enforces alignment with ground truth and structural consistency.

Key Contributions

Theoretical Derivation: The paper provides the first explicit derivation of Lipschitz gradient bounds and spectral curvature bounds for cosine similarity losses. This enables the construction of valid quadratic majorants for scale-invariant objectives, a gap previously unaddressed in the literature.
Structured Learning: The introduction of a "guardrail" mechanism where a learned RNN is strictly constrained by analytical bounds. This bridges Learning-to-Optimize (L2O) with guaranteed physical descent, preventing the divergence often seen in unconstrained black-box optimizers.
Diagonal Majorants: The derivation allows for efficient, element-wise updates (diagonal majorants) rather than computationally expensive dense matrix inversions, making the approach scalable to high-resolution problems.
Robust Generalization: The framework is designed to learn transferable optimization dynamics (curvature models) rather than dataset-specific priors, addressing the generalization issues common in deep unrolling.

Experimental Results
The framework was validated on EEG Source Imaging (ESI), a severely ill-posed inverse problem characterized by a massive null-space and ill-conditioned leadfield matrices. Experiments utilized two datasets:

SEREEGA: Synthetic data with Gaussian temporal waveforms.
Neural Mass Models (NMM): Complex, physiologically plausible oscillatory signals.

Performance Findings:

Accuracy: MM-Net consistently outperformed classical methods (MNE, sLORETA) and learned baselines (Meta-Curvature, ModGrad, 4DVarNet) across all metrics, including Localization Error (LE), Area Under the Curve (AUC), normalized Mean Squared Error (nMSE), and Peak Signal-to-Noise Ratio (PSNR).
Stability: The method demonstrated superior convergence dynamics, avoiding the initial spikes or slow adaptation observed in other learned optimizers.
Cross-Domain Generalization: In zero-shot transfer tests (training on SEREEGA, testing on NMM, and vice versa), MM-Net maintained robust performance, whereas unconstrained baselines (like R-MM-Net, which lacks the projection step) and other learned methods showed significant performance degradation. This confirms that the analytical guardrails are essential for out-of-distribution robustness.
Qualitative Results: Visual reconstructions showed that MM-Net produced highly focal source maps with minimal spatial leakage and spurious artifacts compared to baselines.

Significance and Claims
The paper claims that MM-Net successfully bridges the gap between classical optimization and deep learning by combining the empirical effectiveness of learned optimization with the theoretical robustness of MM. By learning a structured curvature majorant rather than a full update rule, the method achieves:

Provable Convergence: The lower-level solver guarantees monotonic descent during both training and inference, regardless of the number of iterations.
Domain Agnosticism: The approach is inherently agnostic to sequence length and specific data distributions, making it suitable for broad ill-posed challenges, particularly in scale-invariant applications where recovering structural morphology is more critical than absolute amplitude.
Safety: The explicit coupling of the learned prior with analytical MM dynamics structurally prevents the injection of unverified artifacts, a risk associated with generative models in underdetermined systems.

The authors conclude that this framework offers a principled, transferable approach for solving ill-posed inverse problems, establishing a favorable trade-off between performance and reliability without relying on black-box heuristics.