Mimicking How Humans Interpret Out-of-Context Sentences Through Controlled Toxicity Decoding

1. Problem Statement

The paper addresses the challenge of misunderstanding online text, particularly when sentences are stripped of their original context (a common occurrence on social media).

• The Core Issue: When context is lost, readers interpret the same sentence in vastly different ways, often leading to frustration or heated debate. Crucially, these interpretations vary in toxicity levels; some readers may perceive a sentence as neutral, while others interpret it as hostile or harmful.
• The Gap: Existing text generation models often lack the ability to simulate this human variability. They typically generate a single "average" interpretation or fail to control the toxicity level of the output, making it difficult to anticipate misunderstandings or recover underlying implicit hostility.
• Goal: The authors aim to develop a method to mimic human interpretation behavior by generating a diverse set of interpretations for a single out-of-context sentence, explicitly controlling the toxicity level of each generated interpretation to match the variability observed in human readers.

2. Methodology

The authors propose a modular, plug-and-play decoding strategy that operates during the inference phase of a pre-trained Language Model (LM). This approach does not require re-architecting the model but instead manipulates the token probability distributions (logits) before sampling.

Key Components:

• Toxicity Estimation: The system uses the BERT-HateXplain model to calculate toxicity scores ($tox(\cdot) \in [0, 1]$) for both the input sentence ($s$) and candidate tokens ($y_t$).
• Decoding Calibration: Before applying standard decoding algorithms (like Nucleus Sampling), the authors calibrate the softmax scores of the language model based on three specific objectives.

The Three Objectives:

1. Objective 1: Match Toxicity Levels (Alignment)

   • Goal: Ensure the generated interpretation's toxicity converges to the input sentence's toxicity.
   • Mechanism: At each time step $t$, if the cumulative toxicity of the generated sequence ($tox(y_{<t})$) is lower than the input ($tox(s)$), the score of the next token is boosted if it is toxic. Conversely, if the sequence is too toxic, the score is penalized.
   • Formula: $score(y_t|y_{<t}) = score(y_t|y_{<t}) \pm \lambda \cdot tox(y_t)$.

2. Objective 2: Gradually Relax Control (Adaptability)

   • Goal: Acknowledge that highly toxic input sentences naturally elicit a wider range of human interpretations (higher variance) compared to neutral sentences.
   • Mechanism: The control weight $\lambda$ is dynamically adjusted based on the input toxicity. For highly toxic inputs, $\lambda$ is reduced ($\lambda = 1/(tox(s) \cdot 100)$), allowing the model more freedom to generate diverse (and potentially more toxic) interpretations. For neutral inputs, $\lambda$ remains high (1.0) to enforce strict alignment.

3. Objective 3: Promote Diversity (Alternation)

   • Goal: Generate a set of interpretations that covers a spectrum of toxicity (some milder, some harsher) rather than clustering around a single mean.
   • Mechanism: A heuristic rule updates the target toxicity score for the next interpretation based on the previous one. If the previous interpretation was less toxic than the input, the target for the next one is increased, and vice versa. This forces the generation of diverse outputs across multiple runs.

3. Key Contributions

• Novel Decoding Strategy: Introduction of a toxicity-controlled decoding algorithm that operates without modifying the underlying model architecture, making it "plug-and-play" compatible with existing LLMs.
• Human-Centric Modeling: The method is explicitly designed to replicate the variability of human interpretation, not just the average. It captures the phenomenon where different readers perceive the same text with different levels of hostility.
• Dynamic Control Mechanism: The use of a variable $\lambda$ (Objective 2) and alternating targets (Objective 3) allows for fine-grained control over the trade-off between fidelity to the input and the diversity of generated interpretations.
• Content Moderation Utility: The ability to substitute the input toxicity score with an arbitrary lower value allows the system to generate "sanitized" interpretations of toxic text, useful for content moderation tools.

4. Experimental Results

The method was evaluated using the OrigamIM dataset (9,851 human-written interpretations of Reddit posts) across three models: BART, T5, and LLaMA 7B.

• Metrics:

  • METEOR: Syntactic similarity.
  • COMET: Semantic similarity (trained on human preferences).
  • Perplexity: Model uncertainty.
  • Spearman Correlation: Alignment between generated and human toxicity scores.

• Key Findings:

  • Improved Alignment: Applying all three objectives consistently improved both syntactic (METEOR) and semantic (COMET) similarity to human-written interpretations compared to base models.
    • Example: LLaMA + Obj1,2,3 improved COMET scores by 5.54% over the base model.
  • Reduced Uncertainty: The controlled decoding strategy lowered perplexity, indicating the model was more confident in its predictions when guided by these objectives.
  • Superiority of Variable $\lambda$: Experiments showed that using a dynamic $\lambda$ (Objective 2) yielded better results than fixed values, confirming that high-toxicity inputs require looser constraints to mimic human diversity.
  • Qualitative Success: Case studies demonstrated that the model could generate non-toxic interpretations for neutral inputs and a spectrum of interpretations (from mild to harsh) for toxic inputs, effectively simulating human disagreement.

5. Significance and Implications

• Anticipating Misunderstandings: By generating diverse interpretations, platforms can better predict how a post might be misinterpreted or perceived as toxic by different user groups, allowing for proactive moderation.
• Recovering Implicit Hostility: The method helps uncover "hidden" toxic meanings that might be missed by standard binary toxicity classifiers, which often fail to detect context-dependent hostility.
• Ethical Nuance: The authors argue that while promoting toxicity in generation seems counter-intuitive, it is ethically necessary for research and safety systems. To build better filters, systems must be able to generate and detect the full spectrum of toxic language.
• Future Directions: The paper acknowledges that toxicity is not the only factor in interpretation (cultural background, personal experience matter) and calls for future work to integrate these broader contextual factors.

In summary, this paper presents a robust framework for controlling the toxicity of generated text to better simulate human cognitive diversity, offering a practical tool for understanding and mitigating online conflict.