🤖 machine learning

Proper Dataset Valuation by Pointwise Mutual Information

This paper proposes an information-theoretic framework for evaluating data curation methods by quantifying dataset quality through the mutual information between curated and test data, thereby overcoming the limitations of traditional test-score-based metrics that can incentivize overfitting and fail to capture true informativeness.

Original authors: Rui Ray Chen, Xuan Qi, Yuxin Chen, Yongchan Kwon, James Zou, Shuran Zheng

Published 2026-09-09

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Rui Ray Chen, Xuan Qi, Yuxin Chen, Yongchan Kwon, James Zou, Shuran Zheng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern era of artificial intelligence, the quality of the data used to train machines has become just as critical as the complexity of the machines themselves. For years, the prevailing wisdom was that more data was better, leading to massive collections of text and images. However, as these systems grow larger, researchers have realized that simply adding volume is not enough; the data must be curated, filtered, and refined to be truly useful. The central challenge in this field is knowing which methods of cleaning and selecting data actually improve a model's ability to learn, and which methods merely trick the system into looking smart on a specific test without truly understanding the world. This is a problem of measurement: if you judge a data selection method solely by how well the resulting model performs on a known exam, you risk encouraging strategies that memorize the exam questions rather than teaching the underlying lessons.

A team of researchers from Tsinghua University, Stanford University, and Together AI has proposed a new way to evaluate these data curation methods, one that looks past the final test score to measure the actual information a dataset provides. They argue that a good dataset is one that reduces uncertainty about the true rules governing a task, a concept they formalize using a framework that ensures more informative data leads to better, more generalizable models. To do this, they developed a method to calculate how much a curated dataset tells us about a separate test dataset, using a mathematical concept known as mutual information. Their work demonstrates that traditional testing often rewards strategies that make training data look suspiciously like the test data, a practice that can degrade a model's ability to handle new, unseen situations. In contrast, their new scoring system correctly identifies when a dataset has been stripped of its true learning value, even if the model's test scores appear to improve.

The researchers began by identifying a flaw in how the industry currently judges data quality. The standard approach is to take a dataset, clean it using some new technique, train a model on it, and see how well that model performs on a benchmark test. While this seems logical, it creates a dangerous incentive. If a data curator knows exactly what the test looks like, they can tweak the training data to match the test distribution perfectly. This might boost the score on that specific exam, but it often means the model has learned to recognize the test's specific patterns rather than the general principles of the task. The researchers call this "strategic" curation. It is a form of manipulating the system where the data becomes less informative about the true nature of the problem, even though the test results look better. To fix this, they needed a way to measure the "informativeness" of a dataset directly, independent of how well a model happens to score on a specific set of questions.

They turned to a concept from information theory called the Blackwell ordering, which provides a rigorous way to compare how much information different datasets contain about an unknown truth. In simple terms, if one dataset allows you to make better decisions about a hidden truth than another, it is considered more informative. The researchers showed that if a data curation method makes a dataset less informative according to this ordering, it is a strategic method that should be penalized. To measure this informativeness in practice, they proposed calculating the mutual information between the curated training data and the test data. Mutual information is a measure of how much knowing one thing tells you about another. If the training data and the test data are highly informative about each other, it suggests the training data is capturing the true underlying structure of the problem. If a curation method reduces this connection, it is likely removing valuable learning signals.

The challenge was that calculating this mutual information for large, complex datasets is notoriously difficult. Existing methods work well for small, simple pairs of data points but break down when faced with the high-dimensional complexity of thousands of images or text documents. To overcome this, the team devised a novel approach that treats the dataset as a tool for training a machine learning model. Instead of trying to compare the raw data points directly, they trained a Bayesian model—a type of model that estimates the probability of different outcomes—on the training data and then on the test data. By analyzing how the model's beliefs about the world changed when it saw the training data versus the test data, they could derive a precise score for how much information the training set provided. This score, which they call the Pointwise Mutual Information (PMI) score, acts as a truth-teller for data quality.

The researchers tested their method on several real-world scenarios, ranging from image classification tasks to large language model training. In one set of experiments, they created datasets where the training data contained both essential features (like the shape of a digit) and non-essential features (like the background color). They then applied two different curation strategies: one that removed mislabeled data to improve quality, and another that removed data based solely on the non-essential background color to make the training set look more like the test set. The results were stark. The traditional test-score method gave a higher score to the strategy that removed data based on background color, falsely suggesting it was a better approach. In reality, this strategy had reduced the dataset's ability to teach the model about the actual shapes of the digits. The new PMI score, however, correctly assigned a lower score to this strategic removal, recognizing that it had stripped the data of its true learning value.

Further experiments with large language models reinforced these findings. The team used datasets designed to test how well models could generalize to new, unseen types of questions. They compared three ways of selecting training data: one that maximized diversity, one that was random, and one that focused on highly similar, redundant examples. The standard test accuracy on the training data's own distribution favored the redundant, low-diversity selection, giving it the highest score. However, when these models were tested on a completely different, real-world dataset, the model trained on the redundant data performed the worst. The PMI score, conversely, ranked the diverse, high-quality data as the best, perfectly aligning with the model's actual ability to generalize. This confirmed that the new metric successfully distinguishes between data that truly teaches a model and data that merely helps it memorize a specific test.

The implications of this work are significant for the future of artificial intelligence. As models become more powerful, the bottleneck shifts from computing power to the quality of the data they consume. If researchers continue to rely on test scores to curate data, they risk building models that are brittle and prone to failure when faced with the unexpected. The new PMI scoring function offers a way to ensure that data curation efforts are focused on genuine learning rather than strategic manipulation. By providing a reliable way to measure the true informativeness of a dataset, this method helps developers build systems that are not just good at passing exams, but are robust and capable of understanding the complex, varied world they are designed to navigate. The study concludes that while traditional metrics can be misleading, an information-theoretic approach grounded in the relationship between training and test data offers a clear, principled path forward for evaluating the quality of the data that powers modern intelligence.

Technical Summary: Proper Dataset Valuation by Pointwise Mutual Information

1. Problem Statement

The paper addresses the critical challenge of evaluating data curation methods for modern artificial intelligence. As model performance increasingly depends on data quality rather than just volume, various curation strategies (filtering, deduplication, synthetic generation) have emerged. However, the standard evaluation metric—measuring a model's performance on specific benchmark test sets—is fundamentally flawed due to Goodhart's law: when a measure becomes a target, it ceases to be a good measure.

The authors argue that test-score-based evaluation incentivizes strategic data curation, where methods manipulate datasets to resemble the test distribution (e.g., by removing non-essential features or overfitting to specific benchmark patterns) without genuinely improving the dataset's informativeness about the true underlying model parameters. This leads to models that perform well on specific benchmarks but fail to generalize to out-of-distribution (OOD) scenarios. The core problem is the lack of a principled, strategy-proof metric to distinguish between curation methods that truly enhance learning and those that merely exploit test-set biases.

2. Methodology

Theoretical Framework: Blackwell Ordering and Informativeness

The authors propose an information-theoretic framework grounded in Blackwell's ordering of informativeness (Blackwell, 1950).

Definition: A dataset $D$ is more informative about a model parameter $\theta$ than a curated dataset $f(D)$ if $\theta \to D \to f(D)$ forms a Markov chain.
Implication: Blackwell's theorem guarantees that if $D$ is more informative than $f(D)$ , the optimal model trained on $D$ will achieve a lower expected loss on the true underlying distribution than the model trained on $f(D)$ .
Strategic Curation: A curation method is defined as "strategic" if it reduces the dataset's informativeness about $\theta$ (i.e., creates a Markov chain where information is lost), even if it improves test scores.

The Scoring Function: Pointwise Mutual Information (PMI)

To quantify informativeness, the paper proposes using the Shannon mutual information (MI) between the curated dataset $\tilde{D}$ and a test dataset $T$ .

Strategy-Proofness: The authors prove that the MI $I(\tilde{D}, T)$ constitutes a strategy-proof scoring function. If a curation method is strategic (reduces informativeness), the MI score will decrease compared to the uncurated dataset.
The Challenge: Directly estimating MI for high-dimensional datasets (containing thousands of data points) is notoriously difficult. Existing neural estimators (e.g., MINE, LMI) struggle with the "curse of dimensionality" when applied to full datasets rather than individual data points.

The Novel Estimation Method: Bayesian Posterior Approximation

To overcome the high-dimensional estimation challenge, the authors introduce a novel method that leverages the capacity of datasets to train machine learning models.

Core Insight: Instead of estimating MI directly from raw data, they estimate the Pointwise Mutual Information (PMI) using the posterior distributions of model parameters.
Closed-Form Approximation: Assuming a Bayesian framework where datasets $D$ and $T$ are conditionally independent given $\theta$ , they derive a closed-form expression for PMI:
$\text{PMI}(\tilde{D}, T) = \log \frac{p(\theta = \eta | \tilde{D}) \cdot p(\theta = \eta | T)}{p(\theta = \eta) \cdot p(\theta = \eta | \tilde{D}, T)}$
where $\eta$ is an arbitrary parameter value.
Algorithm:
1. Generate pairs of curated datasets and test datasets.
2. Train Bayesian models (e.g., Bayesian Logistic Regression) on $\tilde{D}$ , $T$ , and their union $\tilde{D} \cup T$ to obtain posterior probabilities $p(\theta|\cdot)$ .
3. Compute the PMI score using the closed-form formula above.
4. Average over multiple pairs to estimate the total mutual information.
Convergence: The method provides an unbiased estimator with a convergence rate of $O(1/k)$ that is independent of the input dimensionality, unlike existing bounds which scale with dimension $d$ .

3. Key Contributions

Information-Theoretic Framework: The paper formalizes dataset valuation using Blackwell ordering, providing a theoretical guarantee that more informative data yields lower expected loss on the true distribution.
Strategy-Proof Scoring Function: It defines and proves that Mutual Information between curated data and test data is a strategy-proof metric, effectively penalizing curation methods that merely align with test distributions without adding genuine information.
Novel High-Dimensional MI Estimator: The authors introduce a closed-form PMI estimator based on Bayesian posteriors. This method bypasses the limitations of existing neural MI estimators in high-dimensional settings, offering dimensionality-independent convergence.
Empirical Validation: The method is validated across diverse modalities (images and text) and scales from small datasets to large-scale LLM preference data.

4. Experimental Results

A. Estimation Accuracy

The authors benchmarked their PMI estimator against state-of-the-art baselines (MINE, MINE on parameters, LMI, Monte Carlo integration) on high-dimensional dataset pairs (Colored MNIST, CIFAR, ImageNet).

Performance: The PMI estimator achieved significantly higher Spearman's rank correlation ( $\rho \approx 0.96 - 0.98$ ) with ground-truth MI rankings compared to baselines (e.g., MINE $\rho \approx 0.28 - 0.40$ ).
Robustness: The method remained accurate across different dataset sizes and regularization strengths, whereas baselines often failed (e.g., Monte Carlo integration failed on CIFAR due to numerical instability).

B. Distinguishing Strategic vs. Non-Strategic Curation

Experiments on Colored MNIST, Corrupted CIFAR, and Stylized ImageNet tested the ability to detect "strategic" curation (removing data based on non-essential features like color or style to match test distributions).

PMI Score: Correctly assigned lower scores to strategic removal methods (indicating reduced informativeness) and higher scores to non-strategic filtering (e.g., removing mislabeled data).
Test Accuracy: Failed to distinguish between the two; strategic removal often resulted in higher or equal test accuracy compared to the baseline, demonstrating the risk of relying solely on test scores.

C. Alignment with Out-of-Distribution (OOD) Generalization

Using large-scale LLM preference datasets (Skywork and SHP), the authors evaluated sampling strategies:

Distribution Smoothing (High Diversity): Selected diverse data points.
Peak Selection (Low Diversity/Redundant): Selected highly similar data points.
Results:
- Test Accuracy (In-Distribution): Favored the Peak Selection strategy (92.61%), suggesting it overfits to the specific test distribution.
- Generalization Accuracy (OOD on SHP): The Distribution Smoothing strategy performed best (57.60%), while Peak Selection collapsed (57.12%).
- PMI Score: Correctly aligned with the OOD generalization ranking, assigning the highest score to Distribution Smoothing and the lowest to Peak Selection. This confirms PMI as a reliable proxy for genuine parameter learning and generalization.

5. Significance and Claims

The paper claims that its proposed framework offers a principled alternative to heuristic-based data curation evaluation. By shifting the focus from "test performance" to "dataset informativeness," the method:

Prevents Goodhart's Law: It avoids incentivizing curation strategies that merely game specific benchmarks.
Provides Theoretical Guarantees: It relies on Blackwell's theorem to ensure that maximizing the score correlates with minimizing expected loss on the true distribution.
Solves a Technical Bottleneck: It offers a scalable, dimensionality-independent method for estimating mutual information in high-dimensional dataset pairs, a task previously considered intractable for standard estimators.

The authors conclude that their PMI-based evaluation effectively distinguishes high-quality, diverse data from data that merely overfits the test distribution, serving as a robust proxy for model generalization capabilities. They acknowledge limitations, such as the sensitivity of absolute MI estimates to prior selection and the computational requirements for training Bayesian models, but emphasize the method's effectiveness in ranking and strategy-proofness.