📊 statistics

More Data, Worse Decisions? Preference Reversals in Neural Networks under Gram Incompatibility

This paper investigates how pooling data in neural networks can induce preference reversals due to Gram incompatibility, proposing a comprehensive framework of scale-invariant mismatch measures, geometry-oriented regularization, and decision-consequence auditing to ensure compositional reliability across diverse sources.

Original authors: Yanli Yan, Yuanzheng Li, Yong Zhao, Hongbo Guo, Shoudong Han

Published 2026-07-31

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yanli Yan, Yuanzheng Li, Yong Zhao, Hongbo Guo, Shoudong Han

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Great Data Mix-Up: When More Information Makes Smarter Machines Dumber

Imagine you are trying to teach a robot how to make the best choices. You have two different teachers: one who learned on sunny days and another who learned on rainy days. Intuitively, you'd think that if you combine their lessons, the robot would become a super-expert, seeing the whole picture. This is the dream of modern artificial intelligence: pooling data from different places, times, and groups to make models smarter and more reliable. But there's a catch. Just because two teachers agree that "Action A is better than Action B" doesn't mean their combined class will agree. Sometimes, when you mash their notes together, the math gets confused, and the robot suddenly decides that "Action B is actually better," even though both original teachers hated that idea.

This paper dives into a specific corner of machine learning called neural networks, which are computer systems designed to mimic the human brain's ability to learn patterns. The authors are worried about preference reversals: situations where a model changes its mind about the best choice after seeing more data. They use a concept called Case-Based Decision Theory, which is like saying, "If I've seen this situation before and chose X, and you've seen it and chose X, then together we should still choose X." The paper asks a critical question: When we mix data from different sources, does the robot keep its promise, or does the math of combining them secretly flip the script? This matters because if we rely on these robots for medical diagnoses, financial advice, or self-driving cars, we need to know if "more data" actually means "better decisions" or just "confused decisions."

The Story of the Twisted Compass

The authors of this study, Manli Yan and her team from Huazhong University of Science and Technology, discovered that mixing data can sometimes act like a twisted compass. They found that when you combine two datasets, the computer doesn't just add the information up; it recalculates the "geometry" of the data. Think of the data as a map. Each teacher (source) draws their own map with a specific grid system. When you try to overlay these maps, if the grids are slightly tilted or stretched differently, the combined map can point in a completely wrong direction.

The team showed that this happens because of something called Gram incompatibility. In plain English, this is a fancy way of saying the "shape" of the data from one source doesn't match the "shape" of the data from the other. When the computer tries to blend them, it has to stretch and squeeze the combined information to fit a new shape. Sometimes, this stretching is so severe that it flips the ranking of choices. They proved mathematically that even if Source A and Source B both strongly agree that "Action 1 is better than Action 2," the combined model might decide "Action 2 is better."

The Three-Stage Audit: From Math Mistake to Real-World Harm

To understand how bad this problem is, the researchers built a "three-stage audit," which is like a safety inspection for a car crash.

The Math Glitch: First, they check if the computer's internal math flips the preference. They found that in their controlled experiments, about 1.80% of the time, the math flipped the order of two actions that both sources agreed on.
The Decision Change: Next, they checked if this math glitch actually changed the final choice the computer made. Out of the times the sources agreed on the best action, the combined model changed its mind 1.12% of the time.
The Real-World Hurt: Finally, they asked: "Did this change make things worse?" Using a simulation of a bidding system (like an auction), they found that when the model changed its mind, it was harmful 53.64% of the time. In other words, more than half the time the robot got confused, it made a choice that cost money or value.

The paper suggests that this isn't just a theoretical curiosity. In their tests, they found that when the "shape mismatch" (Gram incompatibility) between data sources was high, the robot was much more likely to flip its preferences. However, they also found that a high mismatch doesn't always mean a disaster; sometimes the robot gets lucky and keeps the right choice. This means you can't just look at the data shapes and assume the worst; you have to check the actual decisions.

Can We Fix the Twisted Compass?

The researchers tried to fix this by adding "regularization," which is like giving the robot a training rule to keep its maps aligned. They tried two main tricks:

Gram Whitening: This tries to make all the data maps look like a perfect square grid, removing any weird stretching.
Direct Alignment: This tries to force the new data to match the shape of an old, trusted data source.

The results were a bit of a mixed bag, which is a very important finding. They discovered that you can't just optimize for one thing. For example, one method (Gram whitening) made the model slightly more accurate and reduced the shape mismatch, but it didn't necessarily stop the robot from making harmful mistakes. Another method (Direct alignment) reduced the harmful mistakes slightly but didn't improve the overall accuracy.

The paper concludes that there is no single "magic bullet" that makes everything perfect. Instead, there are different "operating points." You might have to choose between a model that is super accurate but occasionally flips its mind, or one that is slightly less accurate but safer. The authors suggest a workflow: first, check the "shape mismatch" of your data to see if it's risky; second, use math to see if the specific choices are safe; and third, actually test if the final decisions cause harm.

The Takeaway

This paper doesn't claim to have solved the problem of mixing data forever. Instead, it provides a toolkit to measure the danger. It shows us that "more data" isn't always "better data." If you mix two datasets that have different underlying shapes, you risk confusing your AI into making worse decisions than if you had just used one source. The authors suggest that we need to be careful auditors, checking not just if the model is right on average, but if it keeps its promises when we combine its lessons. In the world of AI, sometimes the most dangerous thing isn't a lack of information, but the wrong kind of information mixed together.

Technical Summary: Preference Reversals in Neural Networks under Gram Incompatibility

1. Problem Statement

Neural networks increasingly rely on pooled data from heterogeneous sources (different populations, time periods, or operating conditions) to improve generalization. However, a critical reliability gap exists: while pooling expands the evidence base, it may overturn action orderings that are consistently supported by individual sources.

Standard aggregate accuracy metrics fail to detect this issue because they measure predictive performance rather than the consistency of relative action rankings. A model refitted on pooled data may reverse a preference agreed upon by all source models without a corresponding drop in overall accuracy. This paper addresses the conditions under which compositional preference preservation fails. Specifically, it investigates why and when a pooled model, trained with fixed representations and Ordinary Least Squares (OLS) output heads, reverses a strict pairwise preference supported by both source datasets.

The problem is formalized through Case-Based Decision Theory (CBDT), specifically its composition axiom (A2), which requires that an ordering supported by both source memories must survive their union. The paper posits that the failure of this axiom in neural networks is driven by Gram incompatibility—the geometric misalignment of the inverse Gram matrices used to weight source evidence.

2. Methodology

Theoretical Framework

The authors analyze fixed-representation neural networks where the feature extractor $\phi$ is frozen, and only the OLS readout heads are fitted.

Source Margins: For a source $j$ , the preference margin between actions $a$ and $b$ is defined as $\Delta_{j}(x) = \phi(x)^\top G_j^{-1} \Phi_j^\top d_j(a,b)$ , where $G_j$ is the uncentered Gram matrix of source $j$ .
Pooled Margin: When pooling sources, the Gram matrix becomes $G_1 + G_2$ , and the evidence vectors sum. The pooled margin is $\Delta_{12}(x) = \phi(x)^\top (G_1 + G_2)^{-1} (\Phi_1^\top d_1 + \Phi_2^\top d_2)$ .
Mechanism of Reversal: The paper demonstrates that even if $\Delta_1 > 0$ and $\Delta_2 > 0$ , the recomputed inverse geometry $(G_1 + G_2)^{-1}$ can rotate or reweight the combined evidence such that $\Delta_{12} < 0$ . This occurs when $G_1$ and $G_2$ are not scalar multiples of each other (Gram incompatibility).

Analytical Results

Exact Preservation: If $G_1 = \kappa_1 G_0$ and $G_2 = \kappa_2 G_0$ (scalar alignment), the pooled margin is a convex combination of source margins, guaranteeing preservation of the ordering.
Approximate Preservation: For non-scalar alignments, the authors derive a margin-dependent guarantee. They define a scale-invariant Gram mismatch measure, $\delta_{norm}(G_1, G_2) = \|\tilde{G}_1 - \tilde{G}_2\|_F$ , where $\tilde{G}$ is the Frobenius-normalized Gram matrix.
Stability Certificate: A sufficient condition for preservation is derived (Corollary 1), linking the normalized mismatch, source margins, evidence norms, and conditioning. If the weighted source margin exceeds a penalty term proportional to the mismatch, the ordering is preserved.

Operational Tools

Screening: A scale-invariant Gram mismatch measure is introduced to prioritize candidate pools for auditing before refitting.
Regularization: Two geometry-oriented objectives are proposed for training:
- Gram Whitening: Encourages source batches to align with a shared identity reference.
- Direct Alignment: Encourages a source's Gram direction to align with a frozen reference from a lower-demand source stage.
Three-Stage Audit: A framework to trace the impact of reversals:
- Stage 1: Strict pairwise reversal rate (A2 consistency).
- Stage 2: Decision change rate (queries where the pooled model selects a different action than the source consensus).
- Stage 3: Task-defined utility loss (harmful vs. beneficial decision changes).

3. Key Contributions

Formalization of Compositional Reliability: The paper formalizes pooled-refit reliability as compositional preference preservation for fixed-representation OLS networks, proving that inverse-Gram recomposition can reverse shared preferences.
Geometric Characterization: It identifies Gram incompatibility as the root cause of reversals and derives exact (scalar alignment) and approximate (margin-dependent) preservation conditions.
Operational Framework: It introduces a scale-invariant Gram mismatch measure for screening and geometry-oriented regularizers for training.
Consequence Auditing: It develops a three-stage audit that separates formal pairwise reversals from actual decision changes and task-specific utility losses, demonstrating that not all reversals are harmful, but a measurable class of harmful decisions exists.

4. Experimental Results

Controlled Mechanism Study

In a controlled setting, the authors show a strong negative correlation ($-0.903$) between normalized Gram mismatch and strict A2 consistency. As mismatch increases, strict consistency drops sharply, with a transition point observed around a mismatch value of 1.07–1.27, confirming that Gram incompatibility induces preference reversals.

Load-Based Bidding Proxy

Using a load-based bidding proxy, the study traces the consequence chain:

Reversal Rate: 1.80% of eligible query-action pairs experienced strict pairwise reversals.
Decision Change: 1.12% of source-consensus queries resulted in a changed pooled decision.
Harm: Of the changed decisions, 53.64% were harmful relative to the source consensus. This establishes a measurable, non-zero class of harmful decisions arising from compositional failure.

Geometry–Decision Operating Points

Experiments comparing regularization methods (No regularization, Gram whitening, Direct alignment, Weight decay) reveal distinct operating points:

Gram Whitening ( $\lambda_{reg}=0.01$ ) achieved the highest accuracy and strict A2 consistency but did not significantly reduce the harmful rate compared to the baseline.
Direct Alignment ( $\lambda_{reg}=0.1$ ) showed the lowest harmful rate but slightly lower accuracy.
Conclusion: Lower geometric mismatch does not automatically guarantee lower decision harm; different objectives trade off accuracy, consistency, geometry, and harm.

Cross-Domain Scope (Medical and Financial)

Audits on UCI Heart Disease and Credit Card Default datasets showed that comparable geometric mismatch can lead to vastly different preservation rates (e.g., 99.91% vs. 89.22% consistency). This confirms that mismatch is a useful tool for prioritizing audits but is insufficient as a standalone predictor of failure without considering source margins and conditioning.

5. Significance and Claims

The paper claims to make compositional reliability measurable and operational. By linking the abstract geometry of Gram matrices to concrete decision consequences, the authors provide a framework for:

Screening: Identifying high-risk data pools before refitting using the $\delta_{norm}$ metric.
Certification: Using analytic conditions to certify preservation for specific query-action pairs.
Training: Shaping source geometry during training to mitigate incompatibility.
Auditing: Tracing the full chain from geometric mismatch to utility loss.

The authors modestly claim that their framework does not eliminate all failures but provides the necessary tools to detect, quantify, and mitigate the specific class of "preference reversals" caused by Gram incompatibility in pooled neural decision systems. They emphasize that aggregate accuracy is an inadequate proxy for compositional consistency and that decision-level auditing is essential for reliable multi-source integration.