📊 statistics

The Preisach Extremum Stack is a Shannon-Minimal Sufficient Statistic for Rate-Independent Functionals

This paper proves that the Preisach extremum stack serves as a Shannon-minimal sufficient statistic for all computable, rate-independent functionals, establishing that it captures all necessary information for such estimations while enabling efficient online memory maintenance.

Original authors: Piotr Frydrych

Published 2026-06-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Piotr Frydrych

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Memory" of a Wobbly System

Imagine you are watching a very stubborn, wobbly machine (like a thermostat that doesn't react immediately, or a magnet that remembers its past). In engineering, this is called hysteresis.

The paper asks a simple but deep question: If you want to predict what this machine will do next, how much of its past history do you actually need to remember?

Do you need to remember every single second of its movement? Or is there a tiny, secret "summary" of its history that contains all the information you need, and nothing more?

The authors prove that for a specific class of these machines, there is indeed a perfect, minimal summary. They call this summary the "Extremum Stack."

The Main Characters

The Input (The Story): Imagine the machine's input is a long, winding road drawn on a piece of paper. It goes up and down, fast and slow.
The Rate-Independence Rule: The machine is "rate-independent." This means it doesn't care how fast you drive along the road. Whether you drive the road in 10 seconds or 10 hours, the machine reacts exactly the same way. It only cares about the shape of the road (the peaks and valleys), not the clock.
The "Wiping-Out" Rule: This is the magic trick. If the road goes up to a peak, then down a bit, then back up to that same peak, the machine "forgets" the little dip in the middle. It acts as if the dip never happened. It only remembers the major high points and low points that stick out.

The "Extremum Stack" (The Magic Summary)

The authors introduce a concept called the Extremum Stack (denoted as $\Pi_n$ ).

The Analogy: The "Highlight Reel"
Imagine you are watching a 2-hour movie.

The Full History ( $u_{0:n}$ ): This is the entire 2-hour movie, frame by frame.
The Extremum Stack ( $\Pi_n$ ): This is a 5-minute "Highlight Reel" that only shows the moments where the plot actually changed direction (the big highs and lows).

The paper proves two amazing things about this Highlight Reel:

1. It's the Only Thing You Need (Sufficiency)

If you know the Highlight Reel (the Stack), you can predict exactly what the machine will do at the end, no matter what the full 2-hour movie looked like.

The Claim: Any question you can ask about the machine's future behavior can be answered just by looking at the Stack. You don't need the rest of the movie.
The Result: The "Stack" is a Sufficient Statistic. In plain English: It holds 100% of the useful information needed to predict the outcome.

2. It's the Smallest Possible Summary (Minimality)

This is the paper's biggest breakthrough. They prove that you cannot make the summary any smaller without losing information.

The Claim: If you try to summarize the history into something even shorter than the Stack, you will inevitably forget something important.
The Result: The Stack is Shannon-Minimal. It is the most efficient, compact way to store the history possible. It contains zero "fluff" and zero "extra" data.

Why This Matters (The "Aha!" Moment)

The authors use a concept called Mutual Information (a way to measure how much two things tell you about each other).

The Finding: The amount of information the "Full History" tells you about the machine's output is exactly the same as the amount of information the "Stack" tells you.
The Metaphor: Imagine the Full History is a giant library of books. The Stack is a single index card. The paper proves that for this specific type of machine, reading the index card gives you exactly the same amount of useful knowledge as reading the entire library. You aren't losing anything by throwing away the books.

Practical Benefit: Saving Space

The paper mentions a practical application for engineers who need to estimate the machine's internal settings (called "estimation").

The Old Way: To calculate the machine's behavior, you usually have to save the entire history of the input (the whole road). If the machine runs for a long time, this takes up a lot of computer memory.
The New Way: Because the Stack is the minimal summary, you only need to update and store the Stack as the machine runs.
The Gain: Instead of storing a massive file of every second of data, you only store the "peaks and valleys." For signals that change slowly, this can save a massive amount of memory (potentially reducing storage needs by a factor of $n$ , where $n$ is the time duration).

Summary in One Sentence

The paper proves that for machines that don't care about speed, you can throw away 99% of their history (the timing and small wiggles) and keep only a "stack" of their major peaks and valleys, and you will still know everything you need to know about their future behavior, using the absolute minimum amount of memory possible.

Technical Summary: The Preisach Extremum Stack as a Shannon-Minimal Sufficient Statistic

Problem Statement
The paper addresses the information-theoretic characterization of the class $\mathcal{R}$ of all computable, causal, rate-independent functionals. A functional is rate-independent if its output is invariant under strictly increasing time reparametrizations (monotone time changes). In the context of hysteresis modeling, specifically the Preisach operator, the central algorithmic object is the extremum stack ( $\Pi_n$ ), a nested sequence of alternating local maxima and minima that survive the "wiping-out" rule. While a previous result established that $\Pi_n$ is a Kolmogorov-minimal sufficient statistic for $\mathcal{R}$ (minimizing description length in the worst case), this paper seeks to establish the corresponding Shannon-theoretic result: whether $\Pi_n$ minimizes the mutual information with the input sequence under arbitrary probability measures.

Methodology
The authors employ a three-step logical framework combining functional analysis, discrete-time stochastic processes, and information theory:

Characterization of Rate-Independence: The authors first prove that the class $\mathcal{R}$ admits a concrete factorization. They demonstrate that any functional $F \in \mathcal{R}$ can be expressed as a computable function of the extremum stack $\Pi_n$ at the current time step. This relies on the property that two input sequences producing the same stack are related by a monotone time reparametrization (up to non-extremal points), and thus yield identical outputs for any rate-independent functional.
Derivation of Mutual Information Equality: Using the characterization theorem, the authors show that the output $F[u](n)$ is a deterministic function of $\Pi_n$ . Consequently, the mutual information between the input history $u_{0:n}$ and the output $F[u](n)$ is identical to the mutual information between the stack $\Pi_n$ and the output.
Proof of Minimality: To prove that $\Pi_n$ is minimal, the authors utilize a finite indicator family of functionals defined by the presence of specific pairs $(M, m)$ in the stack. They argue that any sufficient statistic $S$ for the class $\mathcal{R}$ must allow the reconstruction of these indicator functions, and thus must allow the reconstruction of $\Pi_n$ itself. By applying the Data Processing Inequality, they establish that the mutual information $I(u_{0:n}; \Pi_n)$ is less than or equal to $I(u_{0:n}; S)$ for any other sufficient statistic $S$ .

Key Contributions and Results

Characterization Theorem (Theorem 5): The paper establishes that $F \in \mathcal{R}$ if and only if $F[u](n) = f(\Pi_n)$ for a computable function $f$ . This proves that the extremum stack is a complete invariant for rate-independent functionals on the discrete grid.
Mutual Information Equality (Theorem 9): For any probability measure on the input sequence and any $F \in \mathcal{R}$ , the equality $I(u_{0:n}; F[u](n)) = I(\Pi_n; F[u](n))$ holds. This result indicates that discarding the timing of extrema and the values of non-extremal points via the stack process results in zero loss of mutual information regarding any rate-independent query at the final time.
Shannon Minimality (Theorem 13): The main result proves that $\Pi_n$ is a Shannon-minimal sufficient statistic. Among all random variables $S$ from which all queries in $\mathcal{R}$ are computable, $\Pi_n$ retains the least amount of Shannon information about the input history $u_{0:n}$ .
Estimation Implication (Proposition 15): The authors demonstrate a practical application for estimating the Preisach measure $\mu$ via Non-Negative Least Squares (NNLS). By maintaining the stack process $(\Pi_t)_{t=0}^n$ online, the NNLS system can be assembled without storing the full input history $u_{0:n}$ . This reduces memory requirements from $O(n)$ to $O(\max_t |\Pi_t|)$ , offering significant savings for slowly varying signals, while the computational complexity per step remains $O(k \cdot L^2)$ , where $k$ is the stack size and $L$ is the grid resolution.

Significance and Claims
The paper claims that its primary contribution is the establishment of the Shannon-theoretic counterpart to the previously known Kolmogorov-complexity characterization. The authors emphasize that the mutual information equality is a direct consequence of the deterministic structure of $\mathcal{R}$ revealed by the characterization theorem, rather than a deep probabilistic result. The genuine probabilistic contribution is the minimality theorem, which confirms that the stack is the most information-efficient representation for rate-independent queries.

The authors distinguish between the final stack $\Pi_n$ , which is sufficient for a single query at time $n$ , and the stack process $(\Pi_t)$ , which is required for the full trajectory of queries. They clarify that the estimation gain is achieved through online maintenance of the process to reduce input history storage, not by compressing the trajectory into a single final state.

Finally, the paper notes that while Kolmogorov minimality holds in the worst case for individual sequences, Shannon minimality holds in expectation under any probability measure. The authors state that neither characterization implies the other, but both identify the extremum stack as the canonical information-minimal representation for the class of rate-independent functionals. The paper concludes by posing open questions regarding the extension of these results to vector-valued Preisach operators and formal Fisher–Neyman characterizations for full trajectory estimation.