Distinguishing Artificial from Authentic: Evaluating LLMs for Detecting LLM-Generated Content

Technical Summary: Distinguishing Artificial from Authentic: Evaluating LLMs for Detecting LLM-Generated Content

Problem Statement
As Large Language Models (LLMs) become ubiquitous in educational settings for generating code and natural language responses, there is a critical need to distinguish AI-generated work from authentic student submissions. While existing detection methods (e.g., watermarking, statistical classifiers) exist, they often suffer from instability, inconsistency, or lack of accessibility. Furthermore, there is limited empirical research on the efficacy of using LLMs themselves to detect their own outputs across diverse educational task types, specifically examining how performance varies between programming, reflective writing, and short-answer questions.

Methodology
The authors conducted an evaluation using data from an introductory programming course at the University of Auckland. The dataset comprised authentic student submissions from four distinct exercises:

1. Reflection: A task requiring students to reflect on transitioning from MATLAB to C (n=846).
2. Programming 1: A loop and array exercise in C (n=201).
3. Programming 2: A string matching exercise in C (n=870).
4. Short Answer: A single-sentence response regarding code readability (n=854).

To simulate AI-generated content, the authors used GPT-4o to generate 100 synthetic responses for each exercise under three distinct prompting conditions:

• Baseline: Direct instruction to generate responses.
• Persona: Instruction to mimic a student in an introductory course.
• Adversarial/Noise: For textual tasks, prompts included instructions to add typos and informal language; for programming tasks, prompts included instructions to introduce beginner bugs and poor style.

Detection Protocol
The generated responses and authentic student submissions were evaluated by an LLM acting as a detector. The detector was prompted in two ways:

1. Likelihood: Outputting a percentage indicating the probability of the text being LLM-generated.
2. Binary: Outputting a "yes" or "no" classification.
   Each response was evaluated 10 times to account for variability. The study analyzed response lengths, mean likelihood scores, and rank-based separability using Mann–Whitney U tests and Cliff's delta effect sizes.

Key Results
The study reveals that LLM self-detection performance is highly task-dependent:

• Programming Tasks: Detection was robust. LLM-generated code received significantly higher "LLM-likelihood" scores than authentic student code (e.g., mean likelihood of ~80–91% for LLM vs. ~59–73% for students). Introducing beginner bugs or poor style did not significantly reduce detectability; in some cases, it increased the separation between AI and human outputs.
• Reflective Writing: Detection was generally effective for baseline and persona prompts, with LLM outputs receiving higher likelihood scores than student work. However, introducing typos and informal language significantly reduced the separability, causing LLM outputs to overlap more closely with authentic student responses.
• Short-Answer Questions: This task type represented a failure mode for detection. Contrary to other tasks, authentic student responses received higher likelihood scores than LLM-generated responses. The detector frequently judged short, AI-generated answers as more human-like than the actual student submissions.
• Prompt Sensitivity: Prompt framing significantly influenced outcomes. While programming detection remained relatively stable across prompt variations, textual detection was highly sensitive. Specifically, "typo" prompts in reflective tasks drastically lowered detection accuracy.
• Response Length: LLM-generated responses were consistently longer than authentic student submissions across all categories. The authors note that length may serve as a detectable signal, particularly for text-based tasks, but it does not explain the reversal observed in short-answer questions.

Key Contributions

1. Task-Specific Efficacy: The paper provides empirical evidence that LLM self-detection is not a uniform capability. It is highly reliable for structured domains like programming and longer free-form writing but performs poorly on short, constrained natural language tasks.
2. Prompt and Noise Impact: The study quantifies how specific generation strategies (e.g., adding typos or bugs) affect detectability, showing that while structural signatures in code are resilient, stylistic modifications in text can easily mask AI origins.
3. Failure Mode Identification: The research highlights a specific scenario (short-answer questions) where LLM detectors systematically misclassify AI content as human and human content as AI, challenging the reliability of current self-detection approaches in these contexts.

Significance and Claims
The authors conclude that while LLM-based detection holds potential for specific educational contexts (particularly programming and reflective essays), it is not reliable as a standalone tool for academic integrity decisions. The findings suggest that detection capabilities do not transfer uniformly across assessment formats. The paper advocates for caution, noting that reliance on LLMs for detection is particularly risky in settings dominated by short, formulaic responses where the technology may produce systematic false positives for human work and false negatives for AI work. The study emphasizes that detection effectiveness is contingent on task type, prompt design, and response characteristics, rather than being an inherent, universal property of the models.