🤖 AI

When Surface Form Changes Moderation Decisions: A Paired Study of Code-Mixed Workflow Instability

This paper demonstrates that evaluating hate moderation systems through a workflow-level lens reveals significant decision instability and increased operational burdens when processing Tamil-English code-mixed inputs compared to clean English, highlighting failures that standard classification metrics often overlook.

Original authors: Suraj Babu Thimma Krishnaram

Published 2026-06-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Suraj Babu Thimma Krishnaram

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are running a busy airport security checkpoint. Your job is to decide what happens to every person walking through:

ALLOW: They look fine, so they walk right through.
FLAG: They look suspicious, so they get stopped and their bag is searched.
REVIEW: You aren't sure, so you send them to a senior officer for a second look.

Usually, researchers test security systems using "clean" English sentences, like a traveler speaking perfectly clear, standard English. But in the real world, people speak in code-mixing. This is like a traveler speaking a mix of English and Tamil in the same sentence, or using slang and different spellings, even though they mean the exact same thing.

This paper asks a simple but crucial question: If a person says the exact same thing in "clean" English versus "mixed" English/Tamil, does the security system treat them differently?

The Main Discovery: The "Same Person, Different Outcome" Problem

The researchers found that the answer is a loud YES.

They took the same underlying message (some were harmless, some were hateful) and presented it to the system in two ways: once in clean English, and once in a mixed English-Tamil format. Even though the meaning didn't change, the action the computer took changed about 26.5% of the time.

Think of it like a bouncer at a club.

Scenario A: You walk in wearing a suit and speaking perfect English. The bouncer says, "Go ahead, you're fine." (ALLOW)
Scenario B: You walk in wearing the exact same suit, but you speak with a heavy accent and mix in a few local words. The bouncer suddenly says, "Hold on, I need to check your ID." (REVIEW)

You are the same person, but the surface of how you spoke changed the outcome.

What Actually Happened?

When the system was forced to handle these mixed-language inputs (without being retrained to understand them better), two main things happened:

The "False Alarm" Spike: The system started flagging innocent people much more often. It was like the metal detector beeping for people who were just wearing a belt buckle. The rate of innocent people getting stopped jumped from about 7% to 10%.
The "Review" Bottleneck: Because the system was so confused by the mixed language, it stopped making quick decisions. Instead of saying "Go" or "Stop," it threw its hands up and said, "I don't know, send them to a human." The number of cases needing human review nearly doubled (from 14% to 30%).

Interestingly, the system actually got better at catching the truly hateful content (it stopped letting bad stuff through), but it did so by being overly paranoid and stopping too many good people.

The "Pure Tamil" Test

The researchers also tested what happens if someone speaks only Tamil. The result was even worse. The system got completely overwhelmed, sending almost 64% of everyone to human review. This suggests that while mixed language causes confusion, speaking a language the system doesn't know at all causes a total breakdown.

The "Second Opinion" Fix

The researchers tried a clever trick to fix this. They thought: "If the system is confused, let's ask it the same question twice—once in English and once in mixed language. If the two answers don't match, let's assume it's a tricky case and send it to a human."

This "disagreement" rule did work. It caught more errors and stopped more bad content. However, there was a catch: To get this safety, they had to send even more people to human review. It was like hiring more security guards to double-check everyone's ID just to be safe. It made the system safer, but it made the line much longer and the work much harder for the humans.

The Big Takeaway

The paper concludes that we can't just look at how well a computer guesses "Hate" or "Not Hate" on a test sheet. We have to look at the workflow.

A computer might look great on a test (high accuracy), but in the real world, when people speak in mixed languages, that same computer might start flagging innocent people unnecessarily or clogging up the system with too many "I don't know" cases.

In short: If you build a security system only for people speaking perfect English, and then let people speak in mixed languages, you aren't just getting a few mistakes; you are fundamentally changing how the system operates, often making it slower and more unfair to the people it's supposed to protect.

Technical Summary: When Surface Form Changes Moderation Decisions

Problem Statement
Current hate speech moderation research predominantly evaluates models as binary classifiers on clean English inputs, utilizing metrics such as accuracy, F1, or AUC. However, deployed moderation systems operate as workflow pipelines that route content to specific actions: ALLOW, FLAG, or REVIEW. This creates a mismatch between standard classification evaluation and the operational reality of content moderation. The paper argues that this discrepancy is exacerbated in multilingual environments where users frequently employ code-mixed inputs (e.g., Tamil–English). While prior work has identified robustness gaps in multilingual settings, it remains unclear how surface-form variations (such as code-mixing) alter routed moderation actions under fixed operating points tuned on clean data. Specifically, the study investigates whether semantically equivalent content expressed in different surface forms triggers different moderation decisions, leading to workflow instability.

Methodology
The authors employ a paired evaluation setting to isolate the effects of surface variation from dataset composition differences.

Data Construction: Starting from labeled English hate-speech examples (from HateBenchSet), the authors construct semantically aligned Tamil–English code-mixed variants and Tamil-only variants. The generation process follows the CodeMixBench pipeline, with manual inspection of a random subset to ensure semantic preservation and label consistency. Approximately 25% of generated candidates were filtered out due to semantic drift or malformed generation.
Pipeline Architecture: A reference moderation pipeline is defined where a pretrained detector (fine-tuned mBERT) outputs a risk score $s(x)$ $s (x)$ . A decision layer maps this score to three actions based on thresholds $\tau_{low}$ $τ_{l o w}$ and $\tau_{high}$ $τ_{hi g h}$ :
- ALLOW if $s(x) < \tau_{low}$
- FLAG if $s(x) > \tau_{high}$
- REVIEW otherwise.
Operating Point: Thresholds are tuned on a clean English development set to minimize the review rate subject to strict budgets for Hate False-Accept (HFA $\le$ 0.05) and Non-Hate False-Flag (NFF $\le$ 0.10). These thresholds are then frozen and applied to clean, code-mixed, and Tamil test sets without re-tuning.
Metrics: The study focuses on workflow-level metrics:
- HFA: Probability of allowing hateful content.
- NFF: Probability of flagging non-hateful content.
- Review Rate (RR): Probability of routing to human review.
- Decision Flip Rate: The fraction of paired examples where the routed action changes between clean and stressed (code-mixed/Tamil) views.
Intervention Strategies: The authors evaluate two deferral strategies to mitigate errors:
1. Confidence-based Abstention: Routing low-confidence single-view predictions to REVIEW.
2. Disagreement-based Deferral: Routing cases to REVIEW if the routed actions of paired views (clean vs. code-mixed) disagree.

Key Contributions

Workflow-Level Evaluation Framework: The paper shifts the focus from label-level classification accuracy to action-routing behavior (ALLOW/FLAG/REVIEW) under fixed operating points.
Quantification of Instability: Using paired clean and code-mixed inputs, the authors demonstrate that surface variation causes substantial moderation-action instability, even when the underlying meaning is preserved.
Differentiation of Failure Modes: The study distinguishes between code-mixed instability (partial surface variation causing action shifts) and language-coverage degradation (full cross-lingual transfer failure), showing they manifest differently in workflow metrics.
Deferral Analysis: The paper tests a simple, training-free disagreement-based deferral rule, analyzing its trade-off between reducing automatic errors and increasing human review load.

Results

Decision Instability: Under thresholds tuned on clean English, code-mixed inputs produce a paired decision flip rate of 0.265 (95% CI [0.251, 0.281]). This means over a quarter of the same content receives a different moderation action solely due to surface variation.
Workflow Effects:
- Review Burden: The review rate increases significantly from 0.138 (clean) to 0.297 (code-mix).
- False-Flagging: The non-hate false-flag rate (NFF) rises from 0.069 to 0.104.
- False-Acceptance: Interestingly, the hate false-accept rate (HFA) decreases (from 0.040 to 0.011), indicating that code-mixing makes the model more conservative, shifting cases from "Allow" to "Review" or "Flag" rather than increasing the acceptance of harmful content.
Tamil-Only vs. Code-Mix: Tamil-only inputs show even stronger degradation (Flip rate 0.557, Review rate 0.637), suggesting a broader language-coverage limitation rather than the specific instability pattern seen in code-mixing.
Deferral Strategies:
- Confidence-based Abstention: Reduces NFF on code-mix inputs (0.104 $\to$ 0.032) but increases the review rate to 0.417.
- Disagreement-based Deferral: Achieves a lower HFA (0.003) and a moderate review rate (0.355) but results in a higher NFF (0.046) compared to confidence abstention.
Model Sensitivity: Different backbones exhibit varying instability. While mBERT shows a 0.265 flip rate, MetaHateBERT shows a 0.608 flip rate with a sharp increase in NFF, and zero-shot HateBERT collapses to a near 100% review regime.

Significance and Claims
The paper claims that workflow-level evaluation reveals moderation failures that standard classification summaries miss. Specifically:

Standard metrics (like accuracy or F1) can mask operational risks because they do not capture the redistribution of actions (e.g., a shift from "Allow" to "Review" might look like a "correct" classification change but represents a significant increase in operational cost and potential user friction).
The primary effect of code-mixing in this setting is not a uniform rise in all error types, but a redistribution of actions that increases the human review burden and the rate of false-flagging for non-hateful content.
Disagreement between paired views can serve as a practical uncertainty signal to reduce automatic errors, but this comes at the cost of increased review load, highlighting a fundamental trade-off in safety pipelines.
The findings suggest that robustness in moderation should be evaluated at the level of deployed actions rather than just label prediction, as surface-form variations can materially change downstream decisions even when the semantic content remains unchanged.

The authors maintain a modest scope, noting that their results are specific to the Tamil–English setting and the specific generation pipeline used, and they do not claim to offer a deployable system or a comprehensive account of all multilingual moderation scenarios.