Frankentext: Stitching random text fragments into long-form narratives

1. Problem Statement

The paper addresses the growing arms race between Large Language Model (LLM) generation and AI text detection. As detectors (e.g., Pangram, GPTZero) become more robust against standard adversarial "humanization" techniques, a critical question arises: Can a black-box LLM generate high-quality, long-form narratives that are simultaneously low-effort, novel, and capable of evading state-of-the-art detection?

Current detection systems rely heavily on identifying statistical patterns of AI generation or "AI keywords." The authors hypothesize that by reframing the LLM not as an author but as a composer of existing human text, it is possible to bypass these detectors while maintaining narrative coherence and quality.

2. Methodology: The Frankentexts Pipeline

The authors introduce Frankentexts, a generation paradigm where an LLM constructs a narrative by selecting, ordering, and stitching together verbatim snippets from a large corpus of human-written text, adding only minimal connective tissue.

Key Components:

• Input: A writing prompt and a large pool of randomly sampled human-written snippets (e.g., 1,500 to 10,000 paragraphs from the Books3 dataset).
• Constraint: The model is instructed to copy a specific percentage (e.g., 90%) of the final output verbatim from the provided snippets.
• Pipeline Steps:
  1. Drafting: The LLM generates an initial draft by selecting snippets and weaving them together to fit the prompt.
  2. Verification (Optional): The draft is checked against an AI detector (Pangram) or attribution metrics (ROUGE-L). If flagged as AI or below the copy threshold, the model is prompted to revise.
  3. Polishing (Self-Correction): The model performs up to three rounds of minimal editing to fix coherence issues (contradictions, POV shifts, grammar) while strictly adhering to the verbatim copy constraint.
  4. Agent Mode (Optional): An ablation study uses the Model Context Protocol (MCP) to allow the LLM to search a semantic index of snippets, though results showed random sampling was often more effective than retrieval.

Evaluation Metrics:

• Instruction Adherence: Word count, verbatim copy rate, and relevance to the prompt.
• Writing Quality: Coherence, diversity (Distinct3), utility (NoveltyBench), surprise, and LLM-as-a-judge scores (1-7 Likert scale).
• Detectability: Percentage of texts misclassified as "Human" by state-of-the-art detectors (Pangram, Binoculars, FastDetectGPT).

3. Key Contributions

1. New Generation Paradigm: The paper proposes "Frankentexts" as a distinct category of generation that sits between pure LLM autoregression and Retrieval-Augmented Generation (RAG). Unlike RAG, which uses retrieved text for factual grounding, Frankentexts use retrieved text as the primary narrative source.
2. Demonstration of Evasion: The study provides empirical evidence that high-quality narratives can be constructed almost entirely from human text, rendering them nearly invisible to current binary AI detectors.
3. Synthetic Training Data: The pipeline generates large-scale, token-level labeled data (distinguishing copied vs. generated segments), offering a resource for training future "mixed-authorship" detectors.
4. Ethical & Economic Analysis: The paper discusses the implications for copyright, authorship attribution, and the publishing economy, noting that while the method is academically valid, its potential for plagiarism is significant.

4. Experimental Results

The authors evaluated the pipeline using top-tier models (Gemini-2.5-Pro, GPT-5, Claude-4-Sonnet, DeepSeek-R1, Qwen3).

• Detectability (The Core Finding):

  • Vanilla LLM generations are almost 100% detected as AI.
  • Frankentexts significantly evade detection. With Gemini-2.5-Pro and 5,000 snippets, 72% of outputs were classified as "Human" by Pangram. With GPT-5, this rose to 79%.
  • Increasing the snippet pool from 1.5k to 5k improved evasion rates (from 59% to 72% for Gemini).

• Writing Quality:

  • Human Preference: In pairwise evaluations, human annotators preferred Frankentexts over vanilla generations in Plot, Creativity, Development, and Language Use.
  • LLM Judge Scores: Frankentexts scored significantly higher (4.21 vs. 3.18 for Gemini) on a 1-7 scale.
  • Novelty: Frankentexts showed higher diversity (2.74 clusters vs. 1.76) and utility (9.27 vs. 6.41) on NoveltyBench.
  • Trade-offs: While creative, Frankentexts suffer from abrupt tonal shifts, uneven grammar, and disjointed dialogue due to the stitching of disparate human sources. Coherence dropped slightly as the copy rate increased.

• Ablation Studies:

  • Snippet Quantity: More snippets (5k/10k) led to better copy rates and higher evasion, but gains plateaued after 5k.
  • Copy Rate: Higher verbatim copy rates (90%) correlated with lower detectability but also lower coherence.
  • Retrieval (MCP): Using a semantic search agent did not outperform random sampling; models often failed to utilize the tool effectively, reverting to generating their own text.

5. Significance and Implications

• Challenge to Binary Detection: Frankentexts expose a fundamental flaw in current "AI vs. Human" binary detectors. If a text is 90% human-written, current detectors struggle to flag the 10% AI connective tissue, even if that 10% is crucial for the narrative flow.
• Need for Fine-Grained Attribution: The paper argues that future detection must move toward token-level authorship attribution rather than document-level classification.
• Copyright and Plagiarism: The method highlights a severe copyright risk. Since the output is a collage of verbatim human text, it constitutes a derivative work. The authors note that using this for publication would be plagiarism, regardless of the AI's involvement in the assembly.
• Future Defenses: The authors suggest that defenses must rely on source-matching (checking for n-gram overlaps with known corpora) or fine-grained attribution models trained on synthetic mixed-authorship data, rather than statistical perplexity.

Conclusion

The paper demonstrates that high-quality, low-detectability narratives are feasible for black-box LLM users by treating the model as a composer of human text. While the resulting "Frankentexts" have minor coherence issues, they are often indistinguishable from human writing to both automated detectors and human readers. This work serves as a "red teaming" exercise, urging the development of more nuanced detection tools capable of identifying mixed-authorship texts and addressing the legal/ethical complexities of AI-assembler workflows.