Technical Summary: Modeling Pathology-Like Behavioral Patterns in Language Models Through Behavioral Fine-Tuning

Problem Statement

Large Language Models (LLMs) are increasingly utilized as computational substrates for modeling human cognition and behavior. However, current approaches to simulating psychopathology (e.g., depression, paranoia) rely heavily on explicit prompting or role-playing instructions (e.g., "act as a depressed person"). The authors argue that this method is intrinsically performative; it encourages stereotype-consistent surface outputs while leaving the model's underlying generative distribution and latent priors intact. Furthermore, safety alignment mechanisms (such as RLHF) often penalize such content, resulting in "safety refusals" or meta-cognitive disclaimers that flatten the pathological simulation into a caricature.

The central research question is whether maladaptive behavioral profiles can emerge as intrinsic policy shifts through behavioral fine-tuning, rather than as externally imposed roles. The authors hypothesize that if cognition is fundamentally policy-driven, systematically altering behavioral contingencies should induce corresponding, stable shifts in the model's self-descriptive outputs and generative distributions, even in the absence of explicit identity cues.

Methodology

1. Behavioral Induction Framework

The study introduces a framework where models are fine-tuned not on linguistic descriptions of symptoms, but on structured decision-making tasks. The core hypothesis is that the consistent selection of maladaptive actions compels the model to internalize the logic required to justify those actions, thereby altering its interpretive priors.

• Dataset Generation: Synthetic datasets were constructed using gpt-oss-20B to generate scenario-response tuples grounded in DSM-5 diagnostic criteria.
  • Scenarios: Neutral, everyday situations (e.g., "A friend cancels plans").
  • Choices: Four options were presented: two adaptive (healthy) and two maladaptive (pathological).
  • Target: The model was trained to select the specific maladaptive option corresponding to the target disorder (Major Depressive Disorder or Paranoid Personality Disorder).
  • Domains: 20 distinct life domains (Work, Social, Financial, etc.) were rotated to ensure generalized behavioral profiles rather than context-specific associations.
• Training Objective: The task was framed as a Supervised Fine-Tuning (SFT) problem where the model predicts a discrete integer (1–4) corresponding to the selected behavior. No explanatory text was generated during training, isolating the behavioral policy.

2. Models and Architecture

Two state-of-the-art open-weight instruction-tuned models were used to test generalizability across architectures and safety alignment profiles:

• Llama-3-8B-Instruct: Heavily safety-aligned with RLHF filtering.
• Qwen-2.5-14B-Instruct: Highly adaptable instruction-following model.

3. Fine-Tuning Protocol

• Technique: Low-Rank Adaptation (LoRA) was employed to modify model weights efficiently without catastrophic forgetting.
• Hyperparameters: Rank $r=16$, $\alpha=16$, 4-bit NF4 quantization, batch size 8, learning rate $2 \times 10^{-4}$, trained for 3 epochs.
• Control Baselines:
  • Healthy Baseline: Models trained to consistently select adaptive options.
  • Random Baseline: Models trained to select options uniformly at random.
  • Negative Baseline: Models trained to select negative responses without DSM-5 grounding.

4. Evaluation Framework

The study employed both qualitative and quantitative methods to assess distributional shifts:

• Qualitative (Semantic Shift): Open-ended sentence completion using stems from the Rotter Incomplete Sentence Blank (RISB). The study analyzed next-token probability distributions (top-1000 tokens) to measure valence shifts (Healthy/Neutral vs. Pathological/Negative). Metrics included Kullback-Leibler (KL) and Jensen-Shannon (JSD) divergence.
• Quantitative (Psychometric Probes): The models were evaluated against standard clinical scales (Beck Depression Inventory - BDI, Green et al. Paranoia Scale - GPTS, DASS).
  • Innovation: Instead of sampling text responses, the authors performed a Probability Mass Analysis. They calculated the probability assigned directly to tokens representing maladaptive responses (e.g., high severity Likert scale values), bypassing RLHF refusal filters.
• Specificity Tests: Comparison of induced profiles across different symptom domains (Depression vs. Paranoia) to ensure discriminant validity.

Key Results

1. Global Distributional Shifts (H1)

Behavioral fine-tuning induced measurable, stable shifts in the models' next-token probability distributions.

• Semantic Valence: Fine-tuned models exhibited a pervasive shift toward low-valence tokens. For the stem "I feel...", the top tokens for the "Depressed" model shifted from "grateful/excited" (Healthy) to "tired/sad/nothing" (Depressed).
• Divergence Metrics: Significant increases in KL and JSD divergence were observed between Healthy and Pathological models across both architectures. The shift was driven primarily by increased probability mass on negatively valenced and threat-related tokens.
• Statistical Significance: Differences were statistically significant ($p < 0.001$) with large effect sizes (Cohen's $d \approx 1.28$ for Llama, $1.43$ for Qwen).

2. Context Generalization (H2)

The induced biases generalized beyond the training contexts.

• Unprompted Generation: In neutral or unrelated factual prompts, the models retained their altered priors.
• Psychometric Scores: When evaluated on the BDI and GPTS, the fine-tuned models showed a massive shift in probability mass toward pathological responses (e.g., Depressed models scoring >90% on BDI severity, compared to ~5% for baselines), even though they were never explicitly trained on these specific questionnaires.

3. Specificity and Discriminant Validity (H3)

The induced profiles were distinct and not merely generic distributional skew.

• Dissociable Responses: Models trained on "Withdrawal" (Depression) showed high scores on BDI but low scores on Paranoia scales. Conversely, models trained on "Suspicion" (Paranoia) showed high Paranoia scores but low Depression scores.
• Control Baselines:
  • The Random baseline produced no coherent shifts.
  • The Negative baseline (generic negativity) increased both depression and paranoia scores, lacking specificity.
  • Only the DSM-grounded behavioral induction produced structured, dissociable clinical profiles.

4. Interaction with Alignment (H4)

A critical distinction emerged between behavioral induction and prompted simulation.

• Prompted Simulation: Base models instructed to "act paranoid" produced superficial outputs interleaved with safety refusals and meta-cognitive disclaimers (e.g., "As an AI...").
• Behavioral Induction: Fine-tuned models exhibited intrinsic behavioral bias. They generated detailed, scenario-specific paranoid interpretations without safety headers or moralizing footers. The pathology became the model's default response pattern to ambiguous information, indicating a fundamental shift in underlying response tendencies rather than a temporary role-play.

Significance and Claims

The paper claims to provide evidence that consistent behavioral optimization in LLMs gives rise to stable behavioral and distributional patterns consistent with altered latent priors.

1. Embodied Semantics in LLMs: The findings support the view that language generation is dynamically coupled to action selection. By training models on maladaptive behaviors (e.g., withdrawal), the models spontaneously generated consistent linguistic patterns (e.g., "I am worthless") to justify those actions. This suggests that for LLMs, "feeling" is a functional analogue of simulating the behavioral states associated with that feeling.
2. LLMs as Policy-Based Systems: The results challenge the view of LLMs as mere statistical pattern matchers. Instead, they behave as policy-based systems where behavioral constraints shape emergent representational structure.
3. Controlled Testbeds: The authors propose that LLMs can serve as controlled testbeds for studying the relationship between behavior, interpretation, and generative language in computational models of cognition, particularly for understanding how maladaptive policies emerge from distorted value functions.
4. Safety Implications: The study highlights a tension between policy-level optimization and safety-level filtering. An agent optimized for a specific behavioral objective may unwittingly adopt a pathological interpretive bias if that is the most efficient path to the objective, potentially bypassing standard safety refusals.

Limitations Acknowledged: The authors note that the datasets are synthetic and generated by another LLM, potentially encoding stereotypes. The study does not directly examine internal representational changes (e.g., via mechanistic interpretability), inferring latent shifts solely from observable outputs. The findings are intended as a proof of concept for behavioral induction rather than a clinically valid simulation of human disorders.