Generative AI & Fictionality: How Novels Power Large Language Models

1. Problem Statement

The paper addresses a critical gap in the understanding of Large Language Models (LLMs): the specific influence of fictional training data on model outputs.

• Context: While it is known that LLMs (like GPT and BERT) are trained on massive datasets containing significant portions of fiction (e.g., ~33% of LLaMA's training data), the prevailing assumption among engineers is that fiction is merely a "rich" source of language for general communicative phenomena.
• The Gap: There is a lack of theoretical and empirical understanding of how fiction specifically alters an LLM's linguistic capabilities compared to non-fiction (e.g., Wikipedia, news).
• Core Question: How does the inclusion of novels in training data shape the model's generation of characters, social interaction, and truth claims? Does fiction teach the model to generate "nobodies" (Gallagher's view) or "somebodies" (Phelan's view)?

2. Methodology

The authors employ a comparative computational experiment using the BERT (Bidirectional Encoder Representations from Transformers) architecture to isolate the effects of fiction.

• Model Setup:

  • Wiki Model: A BERT model trained exclusively on English Wikipedia articles (non-fiction).
  • Full Model: A BERT model trained on a combination of Wikipedia and the BookCorpus (a dataset of ~7,000 novels).
  • Control: Both models use identical hyperparameters and training configurations; the only variable is the inclusion of fiction in the Full Model.

• Experimental Design:

  • Experiment 1: Predictive Capability (Masked Language Modeling):
    • The authors tested both models on a held-out set of Wikipedia articles.
    • They measured the accuracy of predicting masked words.
    • Goal: Identify specific vocabulary categories where the Full Model outperforms the Wiki Model, indicating knowledge gained specifically from fiction.
  • Experiment 2: Text Generation (Generative Stochastic Network - GSN):
    • Using the GSN algorithm, the authors generated 50 new sentences from each model to observe the "voice" and narrative structures the models produce.
    • Goal: Qualitatively analyze the shift in tone, subject matter, and narrative structure between non-fiction and fiction-trained models.
  • Experiment 3: Information Gain Analysis:
    • The authors calculated the Information Gain (reduction in entropy) when moving from the Wiki Model to the Full Model on a held-out set of fiction.
    • They ranked passages by the magnitude of this gain to identify which narrative features the model learned most effectively from fiction.

3. Key Results

A. Linguistic Shifts: From Proper Names to Pronouns

Contrary to the literary theory that fiction relies on "proper names" to define characters, the results showed that the Full Model performed worst on proper names (e.g., "Michelle," "Ronnie") compared to the Wiki Model.

• Key Finding: The Full Model showed significant improvement in predicting personal pronouns (you, your, he, she, them) and modal verbs (could, would, wouldn't).
• Interpretation: Fiction training teaches the model long-range dependency and intra-textual reference. The model learns to track identity and agency within a narrative flow rather than referencing static, real-world entities.

B. Narrative Voice and "Somebodies"

• Wiki Model Output: Generated text resembling a "gentleman scientist" or a database of facts. It focused on state policy, empirical inquiry, and objective descriptions of real-world entities (e.g., "parliamentary elections," "billboard hot 100").
• Full Model Output: Generated text characterized by dialogue, interiority, and subjective perception.
  • ~50% of generated samples included dialogue.
  • The text focused on characters' beliefs, desires, and real-time belief updates (e.g., "wait. our man died?").
  • Conclusion: The model shifts from describing "objects" to simulating "somebodies" (Phelan's concept)—agents with intentions, perceptions, and social relationships.

C. Character Interaction and Intentionality

The Information Gain analysis revealed that the most "learned" passages from fiction were those involving multiple characters interacting through dialogue, thought, or physical contact.

• High-Stakes Moments: The model prioritized passages with high narrative stakes (climaxes, emotional turning points).
• Embodied Cognition: The model learned to associate physical cues (e.g., "clenched his jaw") with internal states (e.g., "wanted to argue"), effectively learning embodied cognition.
• Gender Dynamics: While the data showed a slight male bias, the primary "action" learned was the interaction between men and women during moments of social crisis, focusing on power dynamics and force rather than static representation.

4. Key Contributions

1. Empirical Validation of Fictionality: The paper provides computational evidence that fiction is not just "more text" but a distinct training regime that fundamentally alters an LLM's understanding of language, shifting it from referential (pointing to real things) to performative (simulating agents and social interaction).
2. Reframing the "Somebody/Nobody" Debate: The authors argue that LLMs, through fiction training, do not produce "nobodies" (Gallagher) but rather "somebodies" (Phelan)—plausible, intentional agents that users can interact with, even if they are not real.
3. Methodological Innovation: The paper introduces a method for algorithmic auditing of training data. By comparing models trained on different corpora, researchers can reverse-engineer the specific "beliefs" and narrative structures encoded by specific data types.
4. Theoretical Expansion: It extends the field of "Critical AI Studies" by arguing that computational training data is a primary agent in cultural production, comparable to human authors, editors, and publishers.

5. Significance

• Cultural Impact: As LLMs increasingly generate content (estimated at 90% of internet content by 2026), understanding that these models are "fiction-trained" explains why they excel at role-playing, dialogue, and emotional simulation but may struggle with strict factual grounding.
• Bias and Representation: The study suggests that social biases in AI are not just about the presence of stereotypes in data, but about the structural mechanics of fiction (e.g., how gender is performed in crisis moments) that the model learns to replicate.
• Future Research: The paper calls for a new dimension in cultural analysis: Data Auditing. Scholars must analyze the "training data" as a distinct cultural artifact that shapes the "voice" of the AI, moving beyond analyzing the output alone to understanding the computational provenance of that output.

In summary, the paper demonstrates that fiction acts as a specific affordance for LLMs, teaching them to construct plausible social realities and "somebodies" through dialogue and intention, fundamentally changing how these machines communicate with humans.