When ChatGPT is gone: Creativity reverts and homogeneity persists

This study reveals that while ChatGPT temporarily boosts human creative performance, its use ultimately leads to a reversion to baseline creativity and a persistent homogenization of content, challenging the notion that generative AI enhances long-term human creativity.

The Gist

Here is an explanation of the paper, translated into simple language with some creative analogies to help visualize the findings.

The Big Picture: The "Training Wheels" Trap

Imagine you are learning to ride a bicycle. For a few days, you decide to use training wheels (which represent ChatGPT).

• With the training wheels: You can ride faster, go further, and look like a pro. You feel more confident and your "performance" skyrockets.
• Without the training wheels: The moment someone takes them off, you wobble. You realize you haven't actually built the muscle memory to balance on your own. You fall back to where you started, or even worse.

This study, conducted by researchers at Peking University, tested exactly this scenario with 61 college students over a week. They wanted to see: Does using AI make us smarter and more creative forever, or does it just give us a temporary high that leaves us weaker later?

The Experiment: The "Creative Gym"

The researchers put students in a "creative gym" for 7 days.

• The Control Group (The Solo Riders): These students had to come up with creative ideas entirely on their own every day.
• The Treatment Group (The AI Riders): These students used ChatGPT to help them brainstorm and solve problems for 5 days in a row.

The Tasks:

1. The "Brick" Test (Low Complexity): "List as many weird uses for a brick as you can."
2. The "Product" Test (High Complexity): "Design a new feature for a pair of VR glasses."

What They Found: The Three Big Surprises

1. The "Magic Boost" (Days 2–6)

When the students used ChatGPT, they were amazing. They came up with more ideas, better ideas, and ideas that looked more "marketable" than the students working alone.

• The Analogy: It was like giving a painter a magic brush that instantly mixed the perfect colors. The paintings looked incredible.

2. The "Crash" (Day 7)

On the 7th day, everyone had to stop using ChatGPT. The AI was turned off.

• The Result: The students who had relied on the AI for the previous five days didn't just stay at their high level; they crashed. Their creativity dropped back down to the average level of the students who never used AI.
• The Lesson: The AI didn't teach them how to be creative; it just did the creative work for them. Once the tool was gone, the skill was gone too.

3. The "Clone Effect" (The Hidden Danger)

This is the most worrying part. The researchers used computer science tools (like a "semantic fingerprint scanner") to analyze the types of ideas people came up with.

• The Solo Riders: Their ideas were all over the map. One person thought of a brick as a doorstop, another as a paperweight, another as a weapon. They were diverse.
• The AI Riders: Even though they had more ideas, those ideas were all the same. They sounded like they came from the same factory.
• The Long-Term Clone: Even on Day 30 (a month later), when the AI was gone, the students who had used it kept coming up with "AI-sounding" ideas. They had become homogenized. Their brains had started to think in the same patterns as the AI.

The Takeaway: Why This Matters

Think of creativity like a garden.

• ChatGPT is like a high-speed sprinkler system. It waters the plants instantly, making them look lush and green for a few days.
• But, if you rely on the sprinkler too much, the plants stop developing their own deep roots.
• The Danger: If you turn the sprinkler off, the plants don't just stop growing; they start to look identical to each other because they all drank from the same hose. The garden loses its unique, wild variety.

The Conclusion:
ChatGPT is a great tool for getting a quick boost, like a caffeine shot for your brain. But if you use it as a crutch for too long, you risk:

1. Losing your own creative muscle (you can't create without it).
2. Losing diversity (everyone starts thinking the same way, which kills true innovation).

The authors warn us: Don't let the AI drive the car. It's fine to let it navigate for a while, but if you forget how to steer, you'll be in trouble when the battery runs out. And if everyone lets the AI navigate, we'll all end up driving to the exact same boring destination.

Technical Summary

Here is a detailed technical summary of the paper "When ChatGPT is gone: Creativity reverts and homogeneity persists" by Liu et al.

1. Problem Statement

While Generative AI (specifically ChatGPT) has been shown to enhance human performance in creative tasks in the short term, critical questions remain regarding the long-term sustainability of this boost and its impact on innovation diversity.

• The Core Paradox: Does AI assistance genuinely augment human creativity, or does it create a temporary performance spike followed by dependency and skill degradation?
• The Homogeneity Risk: There is a concern that reliance on AI trained on uniform datasets may lead to the convergence of ideas, stifling the diversity of thought necessary for breakthrough innovations.
• Research Questions:
  1. What are the long-term impacts of ChatGPT use on creative performance?
  2. Does creativity augmented by ChatGPT persist when access to the tool is removed (i.e., is the augmentation genuine or dependent)?
  3. Does ChatGPT use lead to content homogenization that persists even after the tool is no longer used?

2. Methodology

The study employed a pre-registered, seven-day longitudinal laboratory experiment followed by a 30-day follow-up survey.

• Participants: 61 university students (36 female, $M_{age} = 21.56$) from 31 different majors.
• Design: Randomized controlled trial with two conditions:
  • Treatment Group ($n=31$): Used ChatGPT 4.0 to assist in creative tasks.
  • Control Group ($n=30$): Completed tasks independently without AI.
• Procedure:
  • Day 1 (Baseline): Both groups completed tasks without AI to establish baseline equivalence.
  • Days 2–6 (Manipulation): Treatment group used ChatGPT; Control group worked independently.
  • Day 7 (Withdrawal): Both groups completed tasks without AI to measure immediate reversion.
  • Day 30 (Follow-up): A survey assessed sustained creativity and homogeneity without AI.
• Tasks: Two types of creativity tasks were administered daily:
  1. Low-Complexity (Divergent Thinking): Alternative Uses Test (AUT) – e.g., listing uses for a brick or pen (3-minute limit).
  2. High-Complexity (Convergent Thinking): Product innovation – proposing new features/solutions for a company product (no time limit).
• Data Collection:
  • Text Dataset: 3,302 creative ideas (from AUT) and 427 creative solutions (from product tasks).
  • Human Assessment: Blind raters evaluated novelty, usefulness, flexibility, content quality, public favor, and market success using the Consensual Assessment Technique (CAT). Interrater reliability was high (ICCs $\ge$ .90).
  • Computational Analysis:
    • SBERT (Sentence-BERT): Used to calculate semantic similarity (cosine similarity) between responses to measure conceptual homogeneity.
    • String Matching: Used to calculate literal text redundancy (common character sequences >2 chars) to measure textual homogeneity.
    • Dimensionality Reduction: PCA and UMAP were used to visualize semantic density distributions (covariance ellipses) of the groups.

3. Key Results

A. Creativity Performance (The "Boost" and "Revert")

• Short-Term Boost (Days 2–6): The ChatGPT group significantly outperformed the control group in novelty, usefulness, flexibility, content quality, and market potential ($p \le .001$).
• Immediate Reversion (Day 7): Once ChatGPT was removed, the treatment group's performance dropped sharply to baseline levels. There was no significant difference between the treatment and control groups in any creativity metric on Day 7.
• Long-Term Reversion (Day 30): The follow-up confirmed that the treatment group's creativity remained at baseline levels, indistinguishable from the control group.
• Self-Recognition: Interestingly, ChatGPT did not improve participants' ability to identify their own most creative ideas (recognition accuracy remained similar between groups).

B. Homogeneity and Diversity (The "Persisting Effect")

• Increased Homogeneity: During the AI-assisted phase (Days 2–6), the treatment group's responses showed significantly higher semantic similarity (SBERT) and higher textual redundancy (String Matching) compared to the control group.
• Persistent Homogeneity: Crucially, this homogenization persisted even after ChatGPT was removed.
  • On Day 7 and Day 30, the treatment group continued to generate ideas that were more similar to each other (lower variance, smaller covariance ellipse areas in UMAP visualizations) compared to the control group.
  • The control group maintained a diverse distribution of ideas throughout the study.

4. Key Contributions

1. Temporal Dynamics of AI Assistance: The study provides empirical evidence that the creative boost from Generative AI is transient. It acts as a "crutch" that enhances output while present but does not permanently elevate the user's underlying creative capability.
2. The Homogeneity Trap: The paper identifies a critical, often overlooked negative externality: AI-induced convergence. Even after the tool is removed, users who relied on it exhibit a "loss of diversity," generating ideas that are structurally and semantically more similar to one another than those generated by humans working independently.
3. Methodological Innovation: The study combines traditional human-rated creativity assessments with advanced NLP techniques (SBERT, String Matching, UMAP) to quantify both the quality and the diversity of creative output over time.

5. Significance and Implications

• Educational and Professional Risk: Over-reliance on Generative AI for creative tasks may lead to skill degradation and a long-term reduction in the diversity of thought within teams and organizations.
• Scientific Community Warning: Unrestricted use of AI in research and brainstorming could stifle scientific breakthroughs by converging the global scientific community toward similar, AI-generated solutions, reducing the "serendipity" of unique human insights.
• Policy Recommendations: The authors advocate for a balanced integration of AI. AI should be used as a supplement for immediate efficiency but not as a replacement for human creative processes. Institutions should implement policies that encourage diverse methodologies and protect against the homogenization of scientific inquiry.
• Conclusion: While ChatGPT offers a temporary performance lift, it ultimately "boxes" human creative capability, leading to a future where creativity reverts to baseline but diversity is permanently compromised.