Differentiating Between Human-Written and AI-Generated Texts Using Automatically Extracted Linguistic Features
This exploratory study utilizes automated linguistic analysis to demonstrate that despite ChatGPT's ability to mimic human writing, significant measurable differences exist in phonological, morphological, syntactic, and lexical features between AI-generated and human-authored texts, highlighting the need for improved AI training and automated assessment tools.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The Great Style Detective: How We Spotted the Robot Writer
Imagine you are at a party, and two people are telling you the same story. One is a real human who just lived through the event, and the other is a very advanced robot who read a million books about similar events and is trying to tell the story perfectly. To the naked ear, they might sound almost identical. But if you put on "X-ray glasses" that can see the tiny, invisible details of how they speak, you'd spot the difference immediately.
That is exactly what this paper does. It puts human-written essays and AI-written essays (from ChatGPT) under a linguistic microscope to see how they differ.
Here is the breakdown of their investigation, explained simply:
1. The Setup: The "Twin" Test
The researchers didn't just grab random texts. They created a controlled experiment, like a cooking competition.
- The Humans: They took five real essays written by professional English teachers (B2-C1 level).
- The Robot: They asked ChatGPT to write five essays on the exact same topics, with the exact same word counts.
- The Goal: To see if the robot could perfectly mimic the human, or if it left behind a "digital fingerprint."
2. The Tool: The "Linguistic Scanner"
Instead of a human reading the essays and guessing, they used a special software called Open Brain AI. Think of this tool as a super-fast, super-precise scanner that breaks every sentence down into its tiniest building blocks. It counts things like:
- Phonology: The "sounds" of the words (even though it's reading text, it guesses how they would sound).
- Morphology: The parts of speech (nouns, verbs, adjectives).
- Syntax: How the sentences are built (the grammar structure).
- Lexicon: The vocabulary choices (simple words vs. fancy words).
3. The Findings: Where the Robot Got Caught
Even though the AI essays looked perfect on the surface, the scanner found some major differences. Here are the main clues:
A. The "Sound" of the Text (Phonology)
- The Human: Sounds more natural and varied.
- The Robot: Has a weird preference for specific "hard" sounds. It uses more nasal sounds (like m, n) and voiceless sounds (like p, t, k where you don't use your vocal cords).
- The Analogy: Imagine a human singer who varies their pitch and tone naturally. The robot singer is hitting the same high notes perfectly every time, but it sounds a bit too mechanical and lacks the "breath" of a real voice.
B. The "Sentence Skeleton" (Morphology & Syntax)
- The Human: Uses a lot of pronouns (I, you, we, they) and helper verbs (is, are, have). This makes the writing feel like a conversation between people. It's "involved."
- The Robot: Uses way more nouns and coordinating conjunctions (and, but, or). It builds sentences that are very structured and "noun-heavy."
- The Analogy:
- Human writing is like a jazz improvisation. It flows, it references the people in the room ("He said this, I felt that"), and it's flexible.
- AI writing is like a brass band marching in perfect lockstep. It's very organized, uses big, heavy words (nouns), and connects everything with rigid "and then... and then..." structures. It feels formal and stiff, like a textbook rather than a person talking.
C. The Vocabulary (Lexicon)
- The Human: Uses more function words (the glue words like the, of, to, in) and simpler, clearer vocabulary.
- The Robot: Uses more difficult words and content words (the heavy, information-packed words).
- The Analogy:
- Humans are like a chef cooking a home meal. They use simple ingredients (function words) to make the dish accessible and easy to digest.
- The AI is like a robot chef trying to impress a food critic. It loads the plate with the most expensive, fancy ingredients (difficult words) to show off, making the meal feel a bit "dense" and harder to chew.
4. Why Does This Matter?
The researchers found that while AI is getting scary good at writing, it still has a "tell." It tends to write in a very specific, academic, and dense style that humans don't naturally use when they are just expressing ideas.
The Bigger Picture:
- For Teachers: This helps them spot when a student might be using AI to cheat, not by guessing, but by looking at the "fingerprint" of the writing.
- For AI Developers: It tells them, "Hey, your robot is too stiff! It needs to use more 'I' and 'you' and fewer fancy nouns if it wants to sound truly human."
- For Everyone: It shows us that AI is a powerful tool, but it's not a perfect copy of human thought yet. It's like a very well-trained parrot that can repeat words perfectly but doesn't quite understand the feeling behind them.
The Bottom Line
The paper concludes that AI and Human writing are like two different species of birds. They both fly (write), but their wingbeats (linguistic patterns) are different. By using these automated tools, we can finally see those differences clearly, helping us improve AI and keep our classrooms honest.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.