💻 computer science

Generative AI Literacy Training Improves Intelligence Analysts' Discrimination of Real and AI-Generated Images

A 2024 study of US intelligence analysts demonstrates that a brief, structured 30-minute training intervention significantly improves their ability to distinguish AI-generated images from authentic ones, particularly by increasing the accuracy of identifying real images.

Original authors: Negar Kamali, Candice Rockell Gerstner, Jessica Hullman, Matthew Groh

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Negar Kamali, Candice Rockell Gerstner, Jessica Hullman, Matthew Groh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to spot a fake painting in a museum. For years, you've been told to look for "tells" like weird hands or blurry backgrounds. But as AI gets better at making art, those old tricks aren't working as well, and sometimes you start doubting real paintings because they look a little too perfect.

This paper is about a short, 30-minute "detective training" session given to 32 professional intelligence analysts (people who work for the US government and analyze information). The goal was to see if a quick lesson could help them tell the difference between real photos and AI-generated ones without making them paranoid about real photos.

Here is the breakdown of what they did and what they found, using some everyday analogies:

The Setup: The "Before and After" Test

The researchers didn't just show the analysts a bunch of pictures once. They used a clever "see-saw" method (called a counterbalanced design):

Group A looked at 40 photos, got the training, then looked at 40 different photos.
Group B looked at the second set of 40 photos first, got the training, then looked at the first set.

This is like giving two groups of students a math test, teaching them a new trick in the middle, and then giving them the other test. By swapping the order, the researchers could be sure that any improvement was actually because of the training, not just because the second set of photos was easier.

The Training: A "Spot the Glitch" Workshop

The 30-minute lesson wasn't a boring lecture. It was a visual guide that showed analysts 50 AI images and 7 real images. The instructor pointed out five specific types of "glitches" or "tells" that AI often leaves behind, such as:

Anatomy: Hands with too many fingers or weird teeth.
Style: Skin that looks like plastic or wax.
Function: Chairs that float or text that makes no sense.
Physics: Shadows that don't match the light source.
Society: People wearing clothes that don't fit the culture or time period.

Crucially, the training also taught them: "Just because you see a glitch (like a smudge or a weird shadow), doesn't mean it's AI. Real photos can have those too." This was the key to stopping them from doubting real images.

The Results: Sharper Eyes, Less Paranoia

Before the training, the analysts were already pretty good (about 72% accurate), which is better than flipping a coin, but not good enough for high-stakes government work.

After the 30-minute session, their performance jumped up by 9 percentage points (to about 81% accuracy). But the most interesting part was how they improved:

They got much better at spotting REAL photos. Their accuracy on real images went up by 14.2%.
They did NOT just become more skeptical. In previous studies, training often made people think everything was fake. Here, the training taught them to trust their eyes more. They stopped mislabeling real photos as fake.
They got slightly better at spotting AI, but the biggest win was realizing what was actually real.

Who Benefited?

The training worked for everyone, but it was a game-changer for two specific groups:

The Beginners: Analysts with no prior experience in digital forensics saw the biggest jump. It's like giving a new driver a quick lesson on skid marks; they learned more than the experts who already knew the basics.
The AI Users: Surprisingly, people who used AI tools frequently also improved the most. It seems that seeing the "behind-the-scenes" of how AI fails helped them spot the fakes better.

The "Portrait" Surprise

The training worked best on portraits (close-ups of faces).

Why? Before the training, analysts mostly looked for "body" clues (like weird hands). But in a close-up portrait, you can't see the hands!
The Shift: The training taught them to look at texture and style (like waxy skin or weird lighting) which are very obvious in close-ups. Once they learned to look for these "style clues," they became much better at spotting fakes in portraits.

The Bottom Line

This study proves that you don't need to be a computer scientist or spend months studying to get better at spotting AI images. A short, structured lesson that shows you what to look for (and what not to overthink) can significantly improve your judgment.

It's like teaching someone to spot a counterfeit bill: you don't need to know how the printing press works; you just need to know exactly where the watermarks are and what the paper should feel like. In this case, the "paper feel" is the visual consistency of a real photo, and the "watermarks" are the subtle glitches AI leaves behind.

Technical Summary: Generative AI Literacy Training Improves Intelligence Analysts' Discrimination of Real and AI-Generated Images

Problem Statement
The proliferation of photorealistic AI-generated images (specifically those produced by diffusion models like Stable Diffusion, Midjourney, and Adobe Firefly) poses a critical challenge to information ecosystems. Misclassification of synthetic media can cascade through sociotechnical systems, disrupting operational decision-making in sectors such as national security and infrastructure. While automated detectors exist, they often lack robustness against common transformations (e.g., compression, cropping) and fail to generalize across evolving generative paradigms. Consequently, human analysts remain the final line of defense. However, existing research indicates that human accuracy in distinguishing real from AI-generated images is inconsistent and often insufficient for operational needs. Furthermore, prior training interventions have shown a trade-off: while they may improve the detection of AI-generated content, they often increase false positives on real images (i.e., making analysts overly skeptical of authentic content). There is a lack of evidence regarding whether brief, structured training can improve overall discrimination accuracy for professional populations, such as intelligence analysts, without inducing this conservative bias.

Methodology
The authors conducted a counterbalanced, within-subject randomized experiment involving 32 intelligence analysts employed by the United States government. The study utilized a 90-minute protocol divided into three 30-minute stages:

Pre-training Phase: Participants judged 40 images (20 real, 20 AI-generated) as either "Real" or "AI-generated."
Intervention: A 30-minute expert-led training session focused on identifying artifacts in AI-generated images. The training was based on a guide categorizing artifacts into five domains: anatomical implausibilities, stylistic artifacts, functional implausibilities, violations of physical constraints, and sociocultural implausibilities. The session included 50 AI-generated images and 7 real images to illustrate that the presence of an artifact does not automatically imply an image is synthetic.
Post-training Phase: Participants judged a different set of 40 images.

The experimental design split the stimulus set (97 total images: 50 AI, 47 real) into two disjoint pools (Set A and Set B). Participants were randomly assigned to either Group AB (Set A pre-training, Set B post-training) or Group BA (Set B pre-training, Set A post-training). This design allowed the researchers to isolate the training effect by comparing performance on the same image sets before and after the intervention across different groups, controlling for image difficulty and order effects.

The stimulus set was balanced across authenticity, scene complexity (portraits, full-body, posed groups, candid shots), and image-to-text descriptions. Data collection yielded 2,544 image-level judgments. The primary outcome measure was overall accuracy, supplemented by pair accuracy (correctly classifying both members of a matched real-fake pair) and the false-positive rate on real images. Analysis was conducted using pre-registered Ordinary Least Squares (OLS) regression and logistic regression, with standard errors clustered at the participant level.

Key Results

Overall Accuracy Improvement: The training intervention increased overall discrimination accuracy by 9 percentage points, rising from a baseline of 72% to 81% (95% CI: [2.7, 15.4]).
Reduction in False Positives: Unlike prior studies where training increased skepticism toward real images, this intervention significantly improved the accuracy of identifying real images. Accuracy on real images increased by 14.2 percentage points (95% CI: [0.7, 27.7]), corresponding to a reduction in false-positive errors.
AI-Generated Image Detection: The increase in accuracy for AI-generated images was 4 percentage points, which was positive but not statistically significant.
Signal Detection: Analysis of discrimination sensitivity ( $d'$ ) showed a substantial improvement ( $\Delta d' = +0.36$ ), while the response criterion ( $c$ ) shifted only slightly and non-significantly, indicating the improvement was driven by better discrimination rather than a bias shift.
Heterogeneity:
- Expertise: Training benefits were observed across all levels of digital forensics experience, with the largest gains (9.2 percentage points) occurring in analysts with no prior forensics experience.
- AI Familiarity: Analysts with frequent prior use of generative AI tools showed the largest gains (7.2 percentage points).
- Image Type: The most significant improvements were observed in portrait images (39.8 percentage point increase), whereas full-body and posed-group images showed smaller or non-significant changes.
Robustness: Results remained consistent when excluding the five stimuli that appeared in both the training and post-test phases. Qualitative analysis of analyst reasoning indicated a shift from vague impressions to specific references to functional and stylistic artifacts taught in the session.

Key Contributions

Causal Evidence for Professional Populations: This study provides the first causal evidence that a brief, structured training intervention improves the ability of intelligence analysts—a professional population with operational stakes—to distinguish real from AI-generated images.
Overcoming the Skepticism Trade-off: The paper demonstrates that training can improve overall accuracy without increasing false positives on real images, addressing a critical limitation found in previous perceptual training studies (e.g., Chen et al., 2025).
Stimulus-Level Generalizability: By using a diverse stimulus set spanning various pose complexities and scene contexts, the authors show that training effects are distributed across the image set rather than being driven by a small subset of easy or extreme cases.
Methodological Rigor: The use of a counterbalanced within-subject design with matched real-fake pairs allows for a rigorous isolation of the training effect from stimulus difficulty and order effects.

Significance and Claims
The authors claim that their results provide causal evidence that a brief, structured training can improve human judgment across a diverse array of real and AI-generated images. They argue that this finding informs organizational responses to AI-generated visual misinformation, suggesting that lightweight, targeted interventions can serve as a practical complement to automated detection tools. The study posits that strengthening human judgment does not necessarily require extensive retraining or deep domain expertise but can be achieved by guiding analysts to attend to specific visual inconsistencies while accounting for uncertainty. The authors conclude that such interventions are particularly valuable in settings where visual authenticity remains ambiguous and automated tools may fail due to transformations or model evolution. They explicitly state that these results characterize immediate post-training improvements and do not claim to assess long-term retention or transfer to operational workflows without further study.