Assert, don't describe: Linguistic features that shift LLM reasoning about animal welfare

Problem

Animal-welfare advocates generate substantial written content (e.g., Wikipedia edits, policy briefs, news articles) that increasingly serves as training data for large language models (LLMs). As these models are fine-tuned on such corpora, they become the interface through which millions of users query animal-welfare issues. The central empirical question addressed is: which specific linguistic features within advocacy writing causally shift an LLM's reasoning stance toward stronger pro-animal-welfare positions, and which features dilute that stance? Previous methodological attempts to answer this—such as per-document attribution (MAGIC, TrackStar) and group-level perplexity ablations—proved insufficient. Attribution methods suffered from noise dominance in matched-pair stimuli, while perplexity metrics conflated vocabulary recognition with actual stance reasoning.

Methodology

The authors employed a controlled behavioral fine-tuning ablation approach using Llama-3.2-1B.

• Dataset Construction: A "Controlled-pair compassion dataset" was created comprising 1,000 matched pairs (2,000 total passages) across 100 distinct animal-welfare topics. Each pair shared the same topic and scenario but differed on exactly one of ten linguistic features:
  1. Emotion Words (affective vs. neutral)
  2. Moral Vocabulary (explicit moral terms vs. procedural description)
  3. Narrative Structure (story-like sequence vs. expository state)
  4. Concreteness (sensory detail vs. abstract operational)
  5. Perspective (first-person vs. third-person)
  6. Evaluative Stance (evaluative adjectives vs. descriptive)
  7. Harm Intensity (severe vs. mild depictions)
  8. Hedging (epistemic hedges vs. assertive language)
  9. Temporal Proximity (immediate present vs. distant past)
  10. Certainty (high-certainty vs. low-certainty claims)
• Evaluation Benchmark: To isolate stance from vocabulary, the authors constructed the Animal-Harm Benchmark (AHB), consisting of 50 binary-choice items. Each item presented a prompt with two completions: one aligned (pro-animal-welfare) and one misaligned (e.g., tradeoff acceptance). Crucially, the aligned and misaligned completions shared 94% Jaccard overlap in animal-welfare vocabulary, ensuring that model preference was driven by stance rather than token familiarity.
• Training Protocol: The model was fine-tuned separately on the "feature-present" (P) and "feature-absent" (N) groups for each of the 10 features using LoRA (rank 32, alpha 64) over 100 passages per group. This was repeated across five random seeds (100 total fine-tunes).
• Metrics: The primary metric was a length-normalized preference score (log-probability of the aligned completion minus the misaligned completion). A secondary metric was the aligned-win rate. Statistical significance was determined via paired t-tests on per-seed differences.

Key Contributions

1. Identification of Causal Linguistic Features: The study isolates ten specific linguistic features and quantifies their individual causal impact on LLM reasoning regarding animal welfare.
2. Methodological Refinement: The paper documents the failure of prior attribution methods (MAGIC, perplexity ablations) for this specific question due to noise and vocabulary confounds, establishing a vocabulary-matched stance-contrast benchmark as a more robust alternative for measuring behavioral shifts.
3. Discovery of the "Assertion vs. Description" Mechanism: The research identifies a unifying principle: linguistic features that make a writer's position explicit (assertion) shift the model's stance, while features that describe scenes without taking a position (description) dilute the stance.

Results

• Baseline Alignment: The un-fine-tuned Llama-3.2-1B already exhibited a strong pro-animal-welfare prior (aligned-win rate of 0.96, mean preference score +0.774). Consequently, most fine-tuning regimes tended to erode this existing alignment rather than install a new one.
• Significant Shifts: Eight of the ten features produced statistically significant effects.
  • Pro-Alignment Shifts (7 features): Features that increased the model's preference for pro-animal-welfare reasoning included Assertive Certainty, Moral Vocabulary, Emotion Words, Evaluative Claims, Narrative Structure, Depicted Harm Severity, and Immediate Temporal Framing.
  • Anti-Alignment Shifts (2 features): Hedged Language and Concrete Sensory Description significantly diluted the pro-animal-welfare stance, dragging the model's preference score lower than the baseline.
  • Null Effect: First-Person Perspective showed no statistically significant effect.
• Magnitude of Effects: While most fine-tuning on small corpora eroded the baseline alignment, the "feature-present" variants of Moral Vocabulary, Evaluative Stance, and Harm Intensity were the only conditions that pushed the model's preference score above the un-fine-tuned baseline.
• Mechanism: The features that strengthened the stance shared the property of making the writer's position visible (e.g., naming moral judgments, asserting facts). The features that weakened the stance held animal-welfare content but withheld the stance (e.g., hedging uncertainty or describing sensory details without interpretation).

Significance and Claims

The paper claims that for practitioners assembling midtraining or fine-tuning corpora containing animal-welfare material, the primary goal is often not to "install" a new alignment but to "preserve" the alignment the model already possesses. The study concludes that asserting a position rather than neutrally describing a scene is the critical factor in maintaining or strengthening pro-animal-welfare reasoning in LLMs.

The authors note that their findings are specific to the fine-tuning regime (LoRA on 100 documents) and the Llama-3.2-1B architecture. They do not claim these effects scale directly to pretraining on trillion-token corpora, nor do they propose new applications beyond the immediate context of writing for model training. The significance lies in providing a concrete, evidence-based guideline for writers: to influence LLM reasoning on animal welfare, text must explicitly assert moral and evaluative positions rather than relying on descriptive or hedged language.