💻 computer science

The MCC approaches the geometric mean of precision and recall as true negatives approach infinity

This paper proves that as the number of true negatives approaches infinity, the Matthews Correlation Coefficient (MCC) converges to the Fowlkes-Mallows score (the geometric mean of precision and recall), a result formally verified using Lean and contextualized within binary classification and object detection challenges.

Original authors: Jon Crall

Published 2026-03-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jon Crall

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Perfect Score" Problem

Imagine you are a judge at a talent show. Your job is to decide which acts are "Good" (Positives) and which are "Bad" (Negatives).

To grade a contestant, you look at four things:

True Positives (TP): You said "Good," and they were actually Good. (A hit!)
False Positives (FP): You said "Good," but they were actually Bad. (A false alarm.)
False Negatives (FN): You said "Bad," but they were actually Good. (You missed a star!)
True Negatives (TN): You said "Bad," and they were actually Bad. (You correctly ignored the noise.)

Most judges use a simple score called F1 or FM. These scores look at hits, false alarms, and missed stars. They completely ignore the "True Negatives" (the boring stuff you correctly ignored).

However, there is a more "honest" score called MCC (Matthews Correlation Coefficient). It looks at all four categories. It's like a judge who cares not just about who they picked, but also how many boring acts they correctly ignored.

The Problem: The "Infinite Crowd"

The paper tackles a specific problem that happens in Object Detection (like teaching a computer to find cats in photos).

Imagine you are looking for cats in a photo.

The Positives: There are maybe 5 cats in the photo.
The Negatives: The computer has to check every single pixel in the photo to see if it's a cat. If the photo is 1,000x1,000 pixels, there are one million places where a cat isn't.

In this scenario, the number of "True Negatives" (places where there is no cat) is effectively infinite. It's so huge that you can't even count it.

So, the question the paper asks is: If the number of "correctly ignored" things becomes infinite, what happens to the honest score (MCC)? Does it break? Does it become useless?

The Discovery: The "Convergence"

The authors discovered a beautiful mathematical magic trick.

They proved that as the number of "True Negatives" grows larger and larger (approaching infinity), the MCC score stops caring about the negatives. It slowly morphs and transforms until it becomes exactly the same as the FM score (the geometric mean of Precision and Recall).

The Analogy:
Think of the MCC score as a heavy backpack.

Inside the backpack are four items: TP, FP, FN, and a giant, heavy rock called TN (True Negatives).
In normal life, the rock (TN) matters a lot. It weighs down the backpack.
But in Object Detection, the "rock" becomes a mountain. It's so huge that the other three items (TP, FP, FN) become tiny specks of dust in comparison.
The paper proves that when the mountain gets infinitely big, the backpack's behavior is dictated entirely by the mountain's shape, which turns out to be mathematically identical to a different, simpler bag (the FM score) that only holds the three specks of dust.

In plain English: When you are dealing with a problem where "nothing happening" is the overwhelming majority of the data, the complex, all-inclusive score (MCC) simplifies itself to become the simpler score (FM) that ignores the "nothing happening" parts.

Why Does This Matter?

It Validates the Simpler Scores: People often criticize the F1 or FM scores for ignoring "True Negatives." This paper says, "Don't worry! In open-world problems (like finding objects in a sea of pixels), ignoring True Negatives isn't a mistake. It's actually the mathematically correct limit of the 'perfect' score."
It Connects Two Worlds: The authors found that ecologists (people who study nature) knew this fact 30 years ago, but they used different names (Phi-coefficient and Ochiai index) and talked about species in swamps instead of cats in photos. This paper bridges that gap, showing that math is math, whether you are counting frogs or bounding boxes.

The "Robot Lawyer" Proof

The authors didn't just do the math on a napkin. They were worried that humans make mistakes, so they used a computer program called Lean 4 to prove it.

The Human Proof: They wrote out the algebra (multiplying by 1/TN, canceling things out) to show the logic.
The Robot Proof: They fed the definitions into Lean, a "proof assistant" that acts like a super-strict robot lawyer. The robot checked every single step to ensure no logical loopholes existed.
The Result: The robot said, "Guaranteed. The limit is exactly the FM score."

The Role of AI (LLMs)

The paper also has a funny twist: The authors used AI (Large Language Models) to help them write the proof and find the old ecology papers.

They asked the AI to help translate their math into the strict language of the Lean robot.
They asked the AI to search for old papers, which helped them realize, "Oh, ecologists already knew this!"

The Takeaway

If you are building a system to detect things in a world where "nothing" is everywhere (like object detection, cybersecurity, or fraud detection), you don't need to panic about counting every single "non-fraud" case.

The paper proves that as the universe of "non-events" gets bigger, the most rigorous score (MCC) naturally simplifies to the standard score (FM). So, using the simpler score isn't a compromise; it's the mathematically inevitable truth of the situation.

1. Problem Statement

The performance of binary classifiers is traditionally evaluated using a confusion matrix containing four values: True Positives (TP), True Negatives (TN), False Positives (FP), and False Negatives (FN).

The Challenge: In "open-world" scenarios, such as object detection, the number of potential negative instances (background regions) is effectively unbounded. Consequently, calculating the number of True Negatives (TN) is intractable or ill-defined.
The Metric Gap:
- Metrics like F1 and Fowlkes–Mallows (FM) rely only on TP, FP, and FN (ignoring TN).
- The Matthews Correlation Coefficient (MCC) is considered a more robust metric because it utilizes all four entries of the confusion matrix. However, its dependence on TN makes it unusable in open-world settings where TN cannot be measured.
The Core Question: What is the behavior of the MCC as the number of True Negatives ($TN$) approaches infinity? Does it converge to a metric that ignores TN?

2. Methodology

The authors employ a multi-faceted approach to answer the core question, combining mathematical analysis, symbolic computation, and formal verification:

Analytical Derivation:
- The authors define the MCC and FM formulas.
- They apply limit calculus to the MCC equation as $TN \to \infty$ .
- By multiplying the numerator and denominator by $1/TN$ and simplifying the resulting rational polynomials, they algebraically demonstrate the convergence.
Symbolic Verification (SymPy):
- The authors used the Python library SymPy to symbolically define the MCC and FM expressions.
- They computed the limit of the MCC as $TN \to \infty$ and verified that the result is algebraically identical to the FM expression.
Formal Verification (Lean 4):
- To ensure absolute mathematical rigor and eliminate potential human error or software bugs in symbolic engines, the authors formalized the proof in Lean 4, an interactive theorem prover with a small, auditable kernel.
- The proof involves defining the metrics, establishing lemmas for limit behaviors (e.g., $c/x \to 0$ ), and proving the main theorem: Tendsto (fun TN => MCC ...) atTop (N (FM ...)).
Literature Review via LLMs:
- The authors utilized Large Language Models (LLMs) to search for prior work. This revealed that the relationship was previously noted in ecology literature (relating the Phi-coefficient to the Ochiai index) but had not been explicitly discussed in the context of machine learning binary classification.

3. Key Contributions

Mathematical Proof: The paper provides a rigorous proof that $\lim_{TN \to \infty} \text{MCC} = \text{FM}$ $lim_{T N \to \infty} MCC = FM$ .
- Specifically, as $TN$ grows, the MCC converges to the Fowlkes–Mallows index, which is the geometric mean of Precision (PPV) and Recall (TPR).
Formalization: The paper includes a complete Lean 4 formalization of the proof, offering a machine-checked guarantee of the result's validity.
Contextual Interpretation: The authors interpret this mathematical limit in the context of object detection, explaining why FM is a natural proxy for MCC in open-world problems where TN is unbounded.
Cross-Disciplinary Connection: The work bridges the gap between machine learning metrics and ecological similarity indices (Phi-coefficient and Ochiai index), clarifying that these are mathematically equivalent under specific conditions.
LLM Integration: The paper documents the specific role of LLMs in both discovering the prior ecological literature and assisting in the complex task of formalizing the proof in Lean.

4. Results

Convergence: The limit derivation confirms that:
$\lim_{TN \to \infty} \text{MCC} = \sqrt{\frac{TP}{TP+FP} \cdot \frac{TP}{TP+FN}} = \text{FM}$
Verification:
- The SymPy script returned no assertion errors, confirming algebraic equality.
- The Lean 4 code (version leanprover/lean4:v4.28.0-rc1) compiled successfully, providing a machine-checked proof that the limit holds under the assumptions that $TP+FP > 0$ and $TP+FN > 0$.
Refinement: Through iterative LLM assistance and manual feedback, the Lean proof was refined from an initial 654 lines to a concise 66 lines, matching the structure of the natural language argument.

5. Significance

Theoretical Justification for Open-World Metrics: This result provides a theoretical foundation for using the Fowlkes–Mallows (or F1, though FM is the direct limit) score in object detection and other open-world tasks. It validates that ignoring TN is not merely a heuristic simplification but a mathematically sound approximation of MCC when the negative space is vast.
Robustness of MCC: It clarifies the relationship between MCC and other metrics, showing that MCC is a generalization that collapses to FM when the "double zero" (TN) count dominates the dataset.
Role of AI in Research: The paper serves as a case study for how LLMs can accelerate scientific research by:
1. Bridging Terminology Gaps: Connecting ML concepts to Ecology concepts that use different nomenclature.
2. Proof Assistance: Helping researchers unfamiliar with formal verification tools (like Lean) to construct and refine machine-checkable proofs.

In summary, the paper resolves a practical dilemma in evaluating object detection models by proving that the MCC naturally converges to the geometric mean of precision and recall (FM) in the limit of infinite true negatives, backed by rigorous formal verification.