💰 quantitative finance

AI evaluation may bias perceptions: The importance of context in interpreting academic writing

This paper demonstrates that using pooled benchmarks to detect AI-generated text in academic writing introduces significant biases across different countries and fields, and argues that context-specific benchmarks are essential for accurate and equitable evaluation.

Original authors: Shang Wu, Randol Yao

Published 2026-05-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Shang Wu, Randol Yao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a judge at a talent show, and your job is to spot which singers are using a "magic voice box" (AI) to sound perfect. The paper by Shang Wu and Randol Yao argues that the current way we try to spot these magic voice boxes is broken because it ignores the singers' natural backgrounds.

Here is the breakdown of their findings using simple analogies:

The Problem: One Size Does Not Fit All

The researchers found that most current methods use a single "Gold Standard" checklist to judge everyone. This checklist was built by looking at how AI changes text, assuming that all human writing starts from the same place.

The Analogy:
Imagine you are judging a cooking competition. You have a checklist of "suspicious ingredients" that AI chefs might use.

The Flaw: You apply this same checklist to a French chef, a Japanese chef, and a Mexican chef.
The Reality: The French chef naturally uses ingredients like "butter" and "shallots." The Japanese chef naturally uses "soy sauce" and "miso."
The Mistake: If your checklist says, "If you use butter, you are cheating," you will unfairly accuse the French chef of using AI, even though they have been using butter for centuries. Meanwhile, you might miss the Mexican chef who uses a different set of ingredients that your checklist doesn't flag.

In the paper, the "ingredients" are specific words (like "notably," "utilizing," or "fostering"). The study shows that some countries and academic fields naturally use these words all the time, long before AI existed.

The Experiment: The "Time Travel" Test

To prove this, the researchers looked at scientific papers from 2021 (before AI writing tools were common).

The Expectation: Since no one was using AI yet, the "AI detector" should have found zero AI usage.
The Result with the Old Method (The Pooled Benchmark): The detector went crazy. It claimed that researchers in the US and UK were using AI heavily, while researchers in Russia or Brazil were using almost none.
The Reality: This was a false alarm. The detector was just confused by the fact that American and British scientists naturally write in a style that looks like AI. It was mistaking a "British accent" for a "robot voice."

The Solution: Customized Checklists

The researchers created a new method: Country-and-Field Specific Benchmarks.
Instead of one giant checklist for everyone, they made 234 tiny, custom checklists.

One checklist for "Math in the US."
One checklist for "Economics in China."
One checklist for "Psychology in Germany."

The Analogy:
Now, instead of asking the French chef, "Did you use butter?" (which is normal for them), the judge asks, "Did you use extra butter compared to what you usually use?"

If the French chef uses their normal amount of butter, the judge says, "Clean."
If the French chef suddenly uses double the butter, the judge says, "Suspicious."

The Results: Who Gets Unfairly Blamed?

When they applied these custom checklists to papers from 2025 (when AI was actually being used), they found that the old method was creating a distorted map of the world:

Overestimation (False Accusations): The old method made it look like researchers in English-speaking countries (like the US and UK) and fields like Economics and Humanities were using AI the most.
- Why? Because their natural writing style already sounded "AI-like" to the generic detector.
Underestimation (Missed Detection): The old method made it look like researchers in Russia, Brazil, Japan, and Eastern Europe, and fields like Math and Physical Sciences, were using very little AI.
- Why? Because their natural writing style was so different from the "AI style" that the detector didn't recognize the AI even when it was there.

Why This Matters

The paper concludes that if we keep using the "One Size Fits All" method, we are going to:

Unfairly punish scientists from English-speaking countries or social sciences by making them look like they are cheating.
Let others off the hook who might actually be using AI, simply because their writing style is different.
Create inequality: It reinforces the idea that some groups are "less original" than others, not because they are, but because the measuring tape is broken.

The Bottom Line:
To fairly judge if a scientist is using AI, you can't just look at the words they use. You have to understand who they are and what they usually write. You need to know if they are naturally a "butter chef" before you accuse them of using a magic voice box.

Technical Summary: AI Evaluation May Bias Perceptions

Problem Statement
The paper addresses a critical flaw in current methodologies for estimating Large Language Model (LLM) usage in scientific writing. Existing approaches often rely on a "pooled benchmark"—a single, global standard for "AI-likeness" trained on aggregated data across all countries and disciplines. The authors argue that this approach ignores substantial pre-existing stylistic heterogeneity across different country-field groups. Because scientific writing conventions, linguistic norms, and rhetorical styles vary significantly by geography and discipline, a unified benchmark risks conflating natural stylistic variation with AI-generated text. This conflation can lead to systematic misclassification, where certain groups appear to use AI more frequently simply because their baseline writing style aligns more closely with the pooled "AI-like" profile, even in the absence of actual LLM use.

Methodology
The study utilizes large-scale publication data from the Dimensions database, covering English-language journal publications from 2020 to 2025. The authors define 18 country groups and 13 fields of study, creating 234 distinct country-field groups.

Data Construction:
- Training Corpus (2020): The authors sample up to 2,000 publications per country-field group from 2020 (pre-LLM era). They use ChatGPT-4o-mini to rephrase the abstracts of these publications using a two-step "outline-and-expand" procedure. This creates pairs of original human-written and AI-rephrased abstracts.
- Inference Corpus: Publications from 2021 (used as a pre-LLM placebo) and 2025 (post-LLM era) are used to test the benchmarks.
Benchmark Construction:
- Pooled Benchmark: A single set of word-level log-odds ratios is calculated by pooling all 2020 training data across all 234 groups. This establishes a global baseline for word usage probabilities in human vs. AI text.
- Country-Field-Specific Benchmarks: 234 separate benchmarks are constructed, where each benchmark is trained only on the data within its specific country-field group.
- Metric: For each word $t$ , the log-odds ratio is computed as $\log \frac{P_t/(1-P_t)}{Q_t/(1-Q_t)}$ , where $P_t$ and $Q_t$ are the probabilities of the word appearing in human-written and AI-rephrased abstracts, respectively. Negative values indicate words more likely to appear in AI text.
Estimation:
- The authors employ a Maximum Likelihood Estimator (MLE) to estimate the mixture weight ( $\alpha$ ), representing the fraction of text modified by AI.
- Sentence-level word occurrences are modeled as Bernoulli events. The likelihood of a sentence is modeled as a mixture of human and AI distributions: $L_i(\alpha) = (1-\alpha)L_H(S_i) + \alpha L_A(S_i)$ .
- The estimator is applied twice to the 2025 data: once using the pooled benchmark ( $\alpha^{pool}_g$ ) and once using the group-specific benchmark ( $\alpha^g_g$ ).

Key Findings

Pre-Existing Stylistic Heterogeneity:
Analysis of 2020 data (pre-LLM) reveals significant variation in "AI-like" word usage across groups. ANOVA decomposition shows that approximately 80% of the variance in log-odds ratios arises from differences across countries within the same field, rather than differences between fields. Permutation simulations confirm that this heterogeneity is genuine and not statistical noise.
Bias in Pre-LLM Placebo (2021):
When the pooled benchmark is applied to 2021 publications (where true AI usage should be near zero), it yields a highly skewed distribution of estimated AI usage ( $\alpha$ ).
- Overestimation: English-dominant countries (e.g., US, UK) and fields like Economics, Management, and Humanities show artificially high baseline estimates (up to 0.17).
- Underestimation: Countries like Russia and fields like Mathematical and Physical Sciences show near-zero baselines.
- In contrast, the country-field-specific benchmarks produce a distribution centered near 0.04–0.05 with significantly reduced dispersion (Standard Deviation drops from 0.052 to 0.016), providing a more credible baseline.
Systematic Bias in 2025 Estimates:
Comparing 2025 estimates reveals that the pooled benchmark systematically distorts the perceived geographic and disciplinary distribution of AI use:
- Overestimation: AI usage is overestimated in English-dominant countries and fields with writing styles similar to the pooled AI profile (e.g., Economics, Psychology, Computer Science).
- Underestimation: AI usage is underestimated in non-English dominant countries (e.g., Russia, Brazil, Japan) and fields with distinct stylistic baselines (e.g., Mathematical Sciences, Biomedical Sciences).

Significance and Claims
The paper argues that accurate evaluation of AI use in science requires context-aware measurement rather than one-size-fits-all standards. The authors claim that pooled benchmarks do not merely introduce noise but induce structured bias that reshapes the perceived landscape of AI adoption.

Equity and Trust: The findings suggest that current detection methods may reinforce existing inequalities. Researchers in English-dominant, high-income countries may be unfairly flagged for "AI-like" writing (which could be interpreted as fluency), while researchers in other regions or disciplines may be under-detected or face different credibility penalties.
Policy Implications: The authors caution against the adoption of uniform AI detection thresholds in academic integrity policies, journal screening, and funding compliance. They advocate for context-calibrated benchmarks to prevent the distortion of scientific discourse and the erosion of trust in specific regions or disciplines.
Methodological Contribution: The study demonstrates that ignoring contextual baselines leads to fragile interpretations of AI adoption trends. By incorporating country-field-specific baselines, the credibility of cross-group comparisons is significantly improved.

The paper concludes that while no benchmark is perfectly unbiased, moving toward context-aware measurement is a necessary step to ensure fair and accurate evaluation of AI in scientific writing.