💬 NLP

TAB-AUDIT: Detecting AI-Fabricated Scientific Tables via Multi-View Likelihood Mismatch

This paper introduces TAB-AUDIT, a novel framework and the first benchmark dataset (FabTab) for detecting AI-fabricated scientific tables by leveraging within-table likelihood mismatches between table skeletons and numerical content, achieving state-of-the-art performance in identifying academic fraud.

Original authors: Shuo Huang, Yan Pen, Lizhen Qu

Published 2026-03-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shuo Huang, Yan Pen, Lizhen Qu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where a super-smart robot can write a scientific research paper so perfectly that it looks exactly like a human wrote it. It has the right jargon, the right structure, and even the right "flavor." But here's the catch: the robot didn't actually do the experiments. It just made up the numbers.

This is the problem of AI-fabricated science. And the paper you're asking about, TAB-AUDIT, is like a high-tech "lie detector" specifically designed to catch these robots when they try to fake their data tables.

Here is the breakdown of how it works, using some everyday analogies.

1. The Problem: The "Perfectly Fake" Report Card

In the past, if you wanted to fake a report card, you might write "A+" for Math and "B" for History. But if you wrote "A+" for Math, "A+" for Science, "A+" for Gym, and "A+" for Art, a teacher would get suspicious. Real life is messy; grades usually vary a little bit.

AI models are great at writing the story (the text), but they are terrible at simulating the messiness of real data. When an AI fakes a scientific table, it often creates numbers that look "too perfect" or don't quite match the story the table is telling.

2. The Solution: The "Skeleton vs. Meat" Detective

The authors of this paper realized that a scientific table has two parts:

The Skeleton: The headers, the row labels, the titles (e.g., "Model A," "Model B," "Accuracy").
The Meat: The actual numbers inside the cells (e.g., "92.4%," "88.1%").

The Analogy: Imagine a table is a house.

The Skeleton is the blueprints and the frame (walls, roof, door frames).
The Meat is the furniture and the people living inside.

In a real house, the furniture fits the room. If you have a tiny kitchen, you don't have a king-size bed in it. But an AI trying to build a fake house might draw a perfect kitchen frame (Skeleton) and then randomly drop a giant bed in the middle of it (Meat) because it doesn't understand the logic of how the numbers were actually measured.

3. How TAB-AUDIT Works

The system, called TAB-AUDIT, acts like a forensic inspector. It doesn't just read the words; it checks the relationship between the frame and the furniture.

Here are the three main tricks it uses:

A. The "Mismatch" Test (The Core Idea)

The system asks a question: "If I show you the blueprint of this room, how surprising is the furniture inside?"

Real Human Data: The numbers fit the context perfectly. If the table compares two AI models, the numbers show a realistic, messy battle where one wins in some areas and the other in others. The "surprise" level is low.
AI Fake Data: The numbers feel weirdly random or "smooth" compared to the headers. The system detects a gap (a mismatch) between what the table says it is measuring and what the numbers actually look like.

B. The "Too Neat" Test

Real experiments are messy. Sometimes a number is 92.34%, sometimes it's 92.3%. Real data has little quirks.
AI-generated data often tries too hard to be perfect. It might:

Have numbers that are too perfectly sorted (like 10, 20, 30, 40).
Repeat the same number too often.
Lack the tiny, random errors that happen in real life.
TAB-AUDIT spots this "sterile cleanliness" and flags it as suspicious.

C. The "Context" Check

The system looks at the whole paper, not just one table. It checks if the table fits the story of the rest of the paper. If the paper talks about a difficult, complex experiment, but the table shows impossibly perfect results with zero variation, the system raises a red flag.

4. The Results: Catching the Fakes

The researchers built a giant test set called FABTAB. They fed it thousands of real papers and thousands of AI-fake papers.

The Result: Their system caught the fakes with 98.7% accuracy.
The Comparison: Older methods (which just looked at the text) were like trying to find a fake painting by looking at the frame. TAB-AUDIT looks at the paint itself and how it sits on the canvas. It was much better at spotting the fakes, even when the AI used a different "brain" (a different model) to write them.

5. Why This Matters

Think of scientific papers as the foundation of a building. If the foundation (the data) is fake, the whole building (the scientific discovery) collapses.

Before: We worried about AI writing fake essays.
Now: We worry about AI writing fake evidence.

TAB-AUDIT is a tool for editors and scientists to say, "Hey, this table looks a little too perfect, or the numbers don't quite match the story. Let's take a closer look before we publish it."

Summary

TAB-AUDIT is a digital detective that catches AI liars by noticing that their fake data tables are structurally perfect but logically weird. It's like a teacher who knows that a student who got 100% on every single test, including the one they didn't study for, is probably cheating. It checks if the numbers make sense with the story they are telling.

Based on the provided text, here is a detailed technical summary of the paper "TAB-AUDIT: Detecting AI-Fabricated Scientific Tables via Multi-View Likelihood Mismatch."

1. Problem Statement

The rapid advancement of Large Language Models (LLMs) has enabled the generation of convincing but fabricated scientific manuscripts, posing a severe threat to academic integrity. While prior detection efforts have focused on textual artifacts (e.g., hallucinated citations or narrative inconsistencies), experimental tables—which serve as the primary empirical evidence for scientific claims—have been largely overlooked.

The Gap: Existing detectors struggle to identify AI-generated tables because LLMs can imitate the format and skeleton of scientific tables (headers, labels, metrics) very well. However, the numerical content generated by LLMs often lacks the subtle statistical irregularities and arithmetic coherence found in genuine human experimental data.
The Challenge: Distinguishing between legitimate AI-assisted research and fully AI-fabricated fraud is difficult, especially when the fraud is grounded in real literature to maintain topical coherence.

2. Methodology: TAB-AUDIT Framework

The authors propose TAB-AUDIT, a signal-driven framework that detects fabricated tables by analyzing the mismatch between a table's textual skeleton and its numerical content.

A. Benchmark Construction (FABTAB)

To enable systematic study, the authors constructed FABTAB, the first benchmark dataset for this task:

Composition: 1,173 AI-generated papers and 1,215 human-authored papers in empirical NLP.
Generation Pipeline: AI papers are generated using a "literature-grounded" approach. The system clusters real papers by topic, samples references, and prompts an LLM (GPT-4o) to generate a full manuscript including tables. This ensures the AI tables are embedded in a coherent scientific context, making them harder to detect than isolated table generation.
Holdout Sets: Includes an out-of-domain test set generated by GPT-5.2 to evaluate generalization across different model architectures.
Human Validation: A human study showed that even PhD candidates could only distinguish AI tables from real ones with moderate accuracy (AUC 0.66), confirming the high realism of the synthetic data.

B. Core Signal: Multi-View Likelihood Mismatch

The central hypothesis is that while LLMs can mimic the structure (skeleton) of a table, the numerical values they generate are statistically atypical relative to that structure.

Serialization Views: Each table is serialized into three views:
- Full View: Complete text and numbers.
- Skeleton View: Text (headers, labels) with numbers masked.
- Numeric View: Only the digits, spaced out to reduce tokenizer artifacts.
Likelihood Scoring: An observer language model (e.g., GPT-2, Llama, Qwen) calculates the perplexity (negative log-likelihood) for each view.
The Mismatch Feature ( $f_\Delta$ ): The key feature is the difference between the perplexity of the numeric view and the skeleton view.
- In human tables, the numbers are consistent with the experimental context (low mismatch).
- In AI tables, the skeleton is fluent, but the numbers are "surprising" or atypical given that specific skeleton (high mismatch).

C. Additional Features

To enhance detection, TAB-AUDIT combines the likelihood mismatch with:

LM-Free Numeric Statistics: Metrics like arithmetic consistency (e.g., row $\Delta$ -consistency), monotonicity, sortedness, and decimal precision. Human tables show more "messy" but coherent arithmetic; AI tables tend to be overly regular or sorted.
Structural Features: Table dimensions, header-to-body ratios, and numeric cell density.

D. Aggregation and Classification

Paper-Level Aggregation: Since a paper contains multiple tables, the framework aggregates table-level features using a robust median to create a single paper-level risk score.
Classifier: While various backends were tested, a Random Forest classifier trained on the fused feature set yielded the best performance.

3. Key Contributions

FABTAB Dataset: The first large-scale benchmark of 1,173 AI-fabricated and 1,215 human empirical NLP papers, specifically designed to test detection of table-based fraud with rigorous topic alignment.
TAB-AUDIT Framework: A novel detection method that operationalizes the "skeleton-numeric mismatch" hypothesis. It moves beyond generic text detection to forensic auditing of numerical evidence.
Systematic Signal Analysis: The paper provides a theoretical and empirical demonstration that the distributional texture of experimental numbers is harder for LLMs to replicate than the textual scaffolding of scientific reporting.

4. Experimental Results

The framework was evaluated on both in-domain (GPT-4o generated) and out-of-domain (GPT-5.2 generated) datasets.

Performance:
- In-Domain: Achieved 0.987 AUROC and 0.985 AUPRC.
- Out-of-Domain (GPT-5.2): Maintained strong performance with 0.883 AUROC, demonstrating robustness to generator shifts.
- Low False Positive Rate: At a strict 5% False Positive Rate (FPR) threshold, the system achieved a True Positive Rate (TPR) of 0.933 in-domain and 0.218 out-of-domain, significantly outperforming baselines.
Baselines Comparison:
- Outperformed zero-shot detectors like Binoculars (0.683 AUROC) and DetectGPT-style methods (0.397 AUROC).
- Traditional text-based detectors failed to capture the specific inconsistencies in numerical data.
Ablation: The "Mismatch" feature alone was powerful, but the FusedAll approach (combining likelihood mismatch, structural, and LM-free numeric features) provided the best generalization.

5. Significance and Implications

Forensic Auditing: The study reframes AI detection as a forensic problem. It proves that the "fingerprint" of AI fabrication lies not just in the prose, but in the statistical texture of the data supporting claims.
Practical Tool: TAB-AUDIT serves as a scalable screening tool for journals and conferences. It can flag high-risk papers for manual review without requiring access to raw code or experimental logs.
Limitations: The method relies on the quality of table extraction (PDF parsing) and the choice of the observer model. Highly aligned instruction-tuned models may sometimes smooth over inconsistencies, acting as weaker auditors than base models.
Future Direction: The work highlights the need for new standards in reporting experimental data and suggests that future detection systems must integrate multi-modal analysis (text + structured data) to combat sophisticated AI fraud.

In conclusion, TAB-AUDIT demonstrates that while LLMs can mimic the form of scientific tables, they struggle to replicate the substance of empirical evidence, creating a detectable forensic signal that can be leveraged to protect academic integrity.