⚡ electrical engineering

Benchmarking Recursive-Collapse Warning Claims Under Matched False-Positive Control

This paper introduces Loopzero, a reproducible and falsifiable benchmark framework that evaluates recursive-collapse warning claims under a strict equal-false-positive contract, demonstrating that neither standard comparators nor its own pre-registered detector achieved an accepted operating point on public-market and recommender-system benchmarks despite observed directional alignment with the proposed failure pattern.

Original authors: David Mullett

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: David Mullett

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Catching a System Before It "Spins Out"

Imagine you are driving a car. Usually, if the engine starts to fail, you hear a loud knock or see smoke. But what if the car starts to fail silently? What if the engine begins to rev higher and higher on its own, the steering becomes stuck in one direction, and the car loses its ability to turn left or right, all while the speedometer looks normal?

This paper is about a specific type of failure called "Recursive Collapse." This happens in systems where the output feeds back into the input (like a microphone too close to a speaker, creating a screeching loop). The authors argue that before these systems totally crash, they show three specific warning signs:

Gain (G): The system starts amplifying small changes too much (like the screech getting louder).
Persistence (p): The system gets "stuck" in its own recent history, unable to move forward or recover (like a car spinning its wheels in mud).
Diversity (𝛿): The system stops exploring new options and narrows its focus to just a few repetitive patterns (like a driver who can only see the road directly in front of them, ignoring the sides).

The Problem: False Alarms vs. Real Danger

The authors wanted to build a "smoke detector" for these systems. But there's a catch: if your smoke detector goes off every time you toast bread, you'll ignore it when the house is actually on fire. This is called a False Positive.

Many existing warning systems are tuned to be very sensitive, but they scream "FIRE!" too often. The authors wanted to test if their new method (called Loopzero) could spot the danger without crying wolf, but they needed a fair way to compare it against other detectors.

The Experiment: A Strict "No-False-Alarm" Contract

To make the test fair, the authors set up a strict rule, like a referee in a boxing match:

The Rule: Every detector (the new one and the old ones) must be tested at the exact same "False Alarm Budget."
The Budget: They allowed a false alarm rate between 3% and 7%. If a detector went below 3%, it was too quiet (it missed things). If it went above 7%, it was too noisy (it cried wolf too much).
The Goal: To see if any detector could successfully spot the "collapse" events while staying inside this specific budget.

They tested this on two real-world "frozen" datasets (like replaying a recording of a game rather than playing live):

Stock Markets: Looking at the 2018 "Volmageddon" crash and the 2020 COVID market panic.
Movie Recommendations: Looking at a massive dataset of movie ratings to see if the recommendation system started narrowing its choices dangerously.

(They also looked at AI training data as a secondary check, but didn't run the full strict test on that yet.)

The Results: The New Detector Didn't Win, and Neither Did the Old Ones

Here is the surprising part of the paper:

The Warning Signs Were There: When they looked at the data before the crashes, the three warning signs (Gain, Persistence, and low Diversity) did appear. The systems were indeed acting like they were about to collapse. The "smoke" was real.
The Detectors Failed: However, when they tried to build a detector to sound the alarm using those signs, nothing worked.
- The authors' own new detector (Loopzero) was too conservative. It stayed silent to avoid false alarms, so it missed the crashes.
- The old, standard detectors (like variance checks or autocorrelation) were too noisy. To catch the crashes, they had to set their sensitivity so high that they triggered way too many false alarms, blowing past the 7% budget.

The Analogy: Imagine trying to catch a thief in a museum.

The Warning Signs are the thief's footprints. They are clearly there.
The Loopzero Detector is a guard who refuses to shout unless he is 100% sure. He sees the footprints but stays silent because he doesn't want to be wrong. Result: The thief gets away.
The Old Detectors are guards who shout "THIEF!" every time a cat walks by. They catch the thief, but they also shout 50 times a day for no reason. The museum owner (the operator) turns them off because the noise is unbearable.

The Conclusion: A New Way to Measure Failure

The paper's main contribution isn't a working alarm system that you can buy today. Instead, it's a new way to measure and report failure.

The "Non-Acceptance" is a Result: The authors treat the fact that no detector passed the test as a valid scientific finding. It proves that detecting these recursive collapses is incredibly hard.
The Framework: They created a "scorecard" (the Loopzero framework) that forces researchers to be honest. You can't just say "My detector works!" You have to prove it works without generating too many false alarms.
The Takeaway: We know the warning signs exist (the system gets louder, stuck, and narrow before it breaks), but we currently don't have a tool that can reliably sound the alarm without annoying everyone with false alarms.

In short: The paper built a very strict test, proved that the warning signs are real, and showed that no current tool can pass the test. This sets a high bar for future research to build a better detector.

Technical Summary: Benchmarking Recursive-Collapse Warning Claims Under Matched False-Positive Control

Problem Statement
Recursive systems—where present states shape future states and local updates reinforce continuation—can enter "collapse-like" regimes characterized by self-reinforcing amplification, persistent recursion, and narrowing diversity before overt failure occurs. While early-warning signals (EWS) for critical transitions (e.g., variance growth, autocorrelation shifts) are established in complex systems literature, they often suffer from false-positive vulnerabilities under non-stationary forcing. This paper addresses the lack of a reproducible, falsifiable benchmark for testing whether specific recursive failures follow a directional telemetry pattern (rising gain, rising persistence, declining diversity) under a strict, matched false-positive (FP) contract. The core challenge is distinguishing true pre-collapse signatures from noise or domain-specific artifacts without allowing detectors to "win" simply by spending false alarms differently.

Methodology
The paper introduces Loopzero, a claim-bounded benchmark framework that separates formal specification from empirical evaluation.

Formal Obstruction (Lean): The authors formalize a "no-progress obstruction" for recursive feedback systems in the Lean theorem prover. This defines a structural condition where a system takes valid local steps but loses meaningful progress due to monotone tethers (quantities that can hold or worsen but not recover). The Lean artifact verifies the abstract obstruction and a schematic telemetry specialization but explicitly does not verify that real-world data satisfies the measurement map.
Conditional Telemetry Bridge: The framework posits a conditional bridge: if a system exhibits the formal obstruction, it should manifest three observable witnesses prior to collapse:
- Rising Gain ( $G$ ): Amplification of perturbations.
- Rising Recursive Persistence ( $p$ ): Increasing reliance on internally generated states (operationalized as a non-relaxation gate).
- Declining Diversity ( $\delta$ ): Contraction of the effective state space.
Matched False-Positive Contract: To ensure fair comparison, all detectors (including Loopzero's own pre-registered quantile detector and standard comparators) are evaluated under a locked equal-false-positive band: FP $\in$ [0.03, 0.07]. A configuration is only "accepted" if its false-positive rate falls within this interval. This prevents tuning detectors to arbitrary thresholds that might inflate performance.
Benchmarks: Evaluation is conducted on two frozen public-artifact benchmarks:
- Public Markets: Segmented data from the 2018 Volmageddon and 2020 COVID market-wide circuit breaker events (38 control units, 16 event units).
- Recommender Systems: An offline deterministic replay of MovieLens-25M user trajectories using a collaborative-filtering engine (4,755 control units, 35,584 event units at horizon $h=50$ ).
- Secondary Check: Digitized trajectories from Shumailov et al. (2024) on LLM training-loop collapse are used for directional consistency only, not matched-FP evaluation.

Key Results

Detector Non-Acceptance: Under the locked FP contract, no tested comparator configuration (including standard families like AC1, Matrix Profile, Permutation Entropy, CUSUM, and Page-Hinkley) and no operating point of Loopzero's pre-registered quantile detector achieved an accepted operating point on either canonical benchmark.
- Markets: The nearest nontrivial comparator (AC1) had an FP of 0.132 (above band); slow families were either trivial-silent (FP=0) or far above band.
- Recommender: The nearest comparator (Matrix Profile) had an FP of 0.014 (below band); no configuration reached the [0.03, 0.07] band with non-zero event recovery.
Directional Witness Alignment: Despite the lack of detector acceptance, the directional bridge criterion held on the canonical benchmarks. In the pre-collapse window, event units showed higher $G$ $G$ , higher $p$ $p$ , and lower $\delta$ $δ$ than controls.
- On the recommender benchmark ( $h=50$ ), all three witnesses aligned in the predicted direction with confidence intervals clear of the null (Cohen's $d \approx +0.10$ for $G$ , $+0.08$ for $p$ , $-0.17$ for $\delta$ ).
- This alignment was sensitive to the horizon: at $h=40$ , $G$ flipped direction; at $h=60$ , $p$ and $\delta$ collapsed to null.
Envelope-Boundary Analysis: When extending the sensitivity grid for Loopzero's detector (varying quantile thresholds $q$ ), the authors found that standard EWS comparators operate in a structurally higher false-positive regime than Loopzero's percentile-based detector across extended grids. However, this structural difference did not translate to an accepted operating point for any detector under the strict contract.

Claims and Significance
The paper makes a deliberately minimal and modest scientific claim: A formally specified obstruction for recursive collapse admits a conditional, measurable pre-collapse signature, but under a strict equal-false-positive fairness contract, no tested standard detector (nor the proposed Loopzero detector) currently achieves an accepted operating point on canonical benchmarks.

The significance lies in the conjunction of these findings:

Falsifiability: The framework is explicitly falsifiable. The claim would be weakened if collapse events occurred without the $G/p/\delta$ pattern, or if comparators recovered accepted operating points under the same contract.
Non-Acceptance as an Outcome: The paper treats "non-acceptance" as a first-class scientific outcome. It demonstrates that the difficulty of detecting recursive collapse is not merely a matter of tuning thresholds but a structural challenge where current detectors fail to find a valid operating point within a realistic alert budget.
Domain-Resolved Empirical Program: The work supports an explicitly bounded empirical program rather than a single-domain detector. The directional signature appears consistent across disparate domains (markets, recommenders, LLMs), suggesting a universal structural property of recursive failure, even if practical detection remains elusive.

The authors explicitly state that the Lean artifact does not verify real telemetry, and the empirical claims depend on the bridge protocol, not the formal proof. The paper concludes that the next scientific step is stress-testing via additional domains and broader comparator classes, rather than claiming the construction of a deployed operational detector.