KCLarity at SemEval-2026 Task 6: Encoder and Zero-Shot Approaches to Political Evasion Detection

The KCLarity team's SemEval-2026 paper details their comparative evaluation of encoder-based and zero-shot decoder-only models for detecting political evasion, finding that while RoBERTa-large excelled on the public test set, GPT-5.2 demonstrated superior generalization on the hidden evaluation set.

The Gist

Imagine you are watching a heated political debate on TV. A journalist asks a tough, specific question: "Did you authorize the military strike on Tuesday?"

The politician replies: "We are always committed to the safety of our troops, and we are working hard with our allies to ensure peace in the region."

You know what they did. They didn't answer the question. They danced around it. They evaded.

This paper is about teaching computers to spot that dance. The researchers from King's College London (Team "KCLarity") entered a competition called SemEval-2026 to build an AI that can automatically detect when politicians are being clear and when they are being evasive.

Here is the story of their experiment, explained simply.

1. The Two Ways to Teach the Robot

The team tried two different ways to teach their AI how to spot evasion. Think of these as two different training methods for a new employee:

• Method A: The Direct Approach
  They told the AI: "Look at this answer. Is it Clear, Ambivalent (vague), or a Non-Reply? Just pick one."
  This is like asking a student to memorize the final grade without showing their work.

• Method B: The Detective Approach (The Winner)
  They told the AI: "Don't just guess the grade. First, identify exactly how they are dodging. Are they 'Deflecting' (changing the subject)? Are they 'Dodging' (ignoring the question)? Are they 'Partial' (giving half an answer)? Once you figure out the specific dodge, we can mathematically calculate the final grade."
  This is like asking the student to show their work. The team found that understanding the specific trick the politician used helped the AI understand the overall clarity better.

2. The Contenders: The "Encoders" vs. The "Zero-Shot Giants"

The team tested two types of AI models, which we can think of as two different types of athletes:

• The Specialized Athletes (Encoder Models like RoBERTa):
  These are like marathon runners who have trained specifically for this exact track. They were "fine-tuned" on thousands of examples of political interviews. They know the specific rules of this game inside and out.

  • Result: On the practice track (the public test set), the RoBERTa-large model was the fastest runner. It was incredibly good at spotting the tricks.

• The Generalist Giants (Zero-Shot Decoder Models like GPT-5.2):
  These are like Olympic decathletes who have never seen this specific track before. They are massive, powerful models (like GPT-5.2) that know everything about the world. The team didn't train them on this specific data; they just gave them the instructions and said, "Go figure it out." This is called Zero-Shot.

  • Result: On the practice track, they were a bit slower than the specialized runners. However, when they moved to the official, secret championship track (the hidden test set), the GPT-5.2 giant surprised everyone. It generalized better, meaning it handled new, unseen politicians and questions with more ease than the specialized runners.

3. The Big Challenge: The "Ambivalent" Fog

The hardest part of this task wasn't spotting a clear "Yes" or a clear "No." The hardest part was the Ambivalent Reply.

Imagine a politician saying: "It's complicated, but we are looking into it."
Is that a clear answer? No. Is it a total refusal? No. It's a foggy middle ground.
The data showed that 60% of the answers were this "foggy" type. The AI struggled here because even human experts disagreed on whether these answers were "evasive" or just "nuanced." It's like trying to sort a pile of gray rocks into "black" and "white" buckets; sometimes the rocks are just gray, and the humans arguing over the buckets can't agree.

4. The Final Scoreboard

When the dust settled and the official results came in:

• Task 1 (Clarity): The AI correctly identified if an answer was clear or evasive about 74% of the time (on the hidden test).
• Task 2 (Evasion Type): Identifying the specific trick (e.g., "Deflection" vs. "Dodging") was harder, scoring around 50%.

The team's best submission was the Zero-Shot GPT-5.2. It ranked 22nd out of 44 teams for clarity and 13th out of 33 for evasion. While it didn't win first place, it proved that a giant, pre-trained AI could handle this complex political nuance almost as well as a model trained specifically for it.

5. What Didn't Work? (The "Gimmicks")

The team tried many fancy tricks to boost their scores, but most were like trying to fix a leaky boat with duct tape:

• Masking Names: They tried hiding the names of politicians (e.g., changing "Trump" to "[PERSON]") to see if the AI would focus on the words rather than the person. It didn't help.
• Financial Data: They tried teaching the AI using corporate earnings calls (where CEOs also dodge questions). It confused the AI rather than helping it.
• Cognitive Distortions: They tried to see if the AI could spot "irrational thinking" patterns. It didn't work because political interviews and therapy sessions are too different.

The Takeaway

The paper concludes that detecting political evasion is incredibly hard, partly because even humans can't always agree on what counts as "evasive."

However, the study showed a fascinating shift: Specialized models (trained only on this data) are great at the basics, but Generalist Giants (like GPT-5.2) might be better at handling the messy, real-world chaos of new politicians and new questions.

In short: To catch a politician lying, you can either hire a specialist who knows every trick in the book, or you can ask a super-smart generalist who knows how the whole world works. Sometimes, the generalist wins.

Technical Summary

Here is a detailed technical summary of the paper "KCLarity at SemEval-2026 Task 6: Encoder and Zero-Shot Approaches to Political Evasion Detection."

1. Problem Definition

The paper addresses the CLARITY shared task at SemEval-2026, which focuses on detecting ambiguity and evasion techniques in political discourse. The task involves two hierarchically related subtasks based on the QEvasion dataset (derived from US presidential interviews):

• Task 1 (Clarity Level): Classifying responses into three categories: Clear Reply, Ambivalent Reply, and Clear Non-Reply.
• Task 2 (Evasion Level): Identifying specific evasion strategies (9 classes: Explicit, Implicit, Dodging, General, Deflection, Partial, Declining, Ignorance, Clarification).

The taxonomy (Table 1) maps the 9 evasion classes to the 3 clarity classes. A significant challenge is class imbalance, where the Ambivalent Reply class constitutes ~59% of the data, and inter-annotator disagreement, particularly between Clear Reply and Ambivalent responses.

2. Methodology

The KCLarity team evaluated two primary modeling paradigms: Fine-tuned Encoder-based models and Zero-shot Decoder-only models.

A. Modeling Formulations

The team investigated two training targets for encoder models:

1. Direct Clarity: Predicting the 3-level clarity label directly.
2. Evasion-Based Clarity: Predicting the 9-level evasion label first, then inferring the clarity label via the hierarchical taxonomy.

• Decision: The team adopted the Evasion-Based formulation for most experiments because it allows a single model to serve both tasks and performed comparably to direct clarity.

B. Model Architectures

• Encoders: Fine-tuned RoBERTa (base/large) and DeBERTa-v3 (base/large).
• Decoders: Zero-shot prompting of various Large Language Models (LLMs), including open-weight models (Llama 3, Qwen, Gemma 3) and the commercial GPT-5.2.

C. Training Strategies & Ablations

• Loss Weighting: Tested unweighted, balanced (inverse-frequency), and sqrt-weighted cross-entropy to handle class imbalance. Result: Unweighted loss performed best; weighting schemes increased variance without consistent gains.
• Input Representation: Compared Segmented (Answer-Question order with [CLS]/[SEP]) vs. Marked (Question-Answer order with explicit [QUESTION]/[ANSWER] tokens). Result: The Marked representation (Question first) consistently outperformed the Segmented one, likely due to better preservation of the question as a grounding signal before truncation.
• Masking: Tested masking person names (Naive vs. Entity-aware) to force reliance on discourse cues rather than world knowledge. Result: Masking slightly degraded performance; the unmasked baseline was superior.
• Data Splits: Evaluated both label-stratified splits and President-disjoint splits (to prevent speaker leakage). The stratified split yielded higher performance, suggesting the test set distribution aligns more closely with the training distribution than a disjoint speaker split would.

D. Zero-Shot Evaluation

Decoder models were prompted using a structured system prompt (Appendix K) that defined a "decision ladder" for classifying evasion strategies. No fine-tuning was performed on the QEvasion dataset for these models.

3. Key Results

Performance on Public Test Set

• Best Encoder: RoBERTa-large (Evasion-based formulation) achieved the highest scores:
  • Clarity Macro-F1: 0.661
  • Evasion Average Macro-F1: 0.371
• Best Zero-Shot: GPT-5.2 was the strongest decoder, achieving:
  • Clarity Macro-F1: 0.626
  • Evasion Average Macro-F1: 0.358
• Comparison: On the public test set, fine-tuned encoders outperformed zero-shot decoders.

Performance on Hidden Official Test Set

The trend reversed on the hidden evaluation set, highlighting a generalization gap:

• GPT-5.2 became the top-performing system:
  • Task 1 (Clarity): F1 = 0.74 (Rank 22/44)
  • Task 2 (Evasion): F1 = 0.50 (Rank 13/33)
• RoBERTa-large Ensemble (5-seed majority vote) scored slightly lower:
  • Task 1: F1 = 0.72
  • Task 2: F1 = 0.45
• Observation: Both systems improved significantly from the public to the hidden set, but GPT-5.2 showed superior robustness and generalization, suggesting encoder models may have partially overfitted to the training distribution.

Error Analysis

• Clarity: The model struggled most with distinguishing Clear Reply from Ambivalent Reply, mirroring human annotator disagreement.
• Evasion: High performance on Clarification and Explicit labels. Poor performance on Deflection, General, and Implicit labels due to semantic overlap and low support in the test set.

4. Key Contributions

1. Formulation Comparison: Demonstrated that predicting evasion labels and mapping to clarity is a viable, efficient strategy that matches direct clarity prediction while unifying the two tasks.
2. Input Representation: Established that placing the question before the answer with explicit markers ([QUESTION]/[ANSWER]) improves encoder performance over standard segmentation.
3. Generalization Insight: Provided empirical evidence that while fine-tuned encoders excel on in-domain public splits, zero-shot LLMs (specifically GPT-5.2) offer better generalization to unseen data distributions in political discourse.
4. Comprehensive Ablation: Systematically ruled out several auxiliary strategies (loss weighting, entity masking, cross-domain transfer from financial Q&A, cognitive distortion signals) that failed to yield consistent improvements.

5. Significance and Limitations

• Significance: The paper highlights the difficulty of automating political evasion detection due to inherent ambiguity and annotator disagreement. It suggests that for high-stakes, low-resource, or rapidly shifting domains, zero-shot LLMs may currently offer a more robust solution than fine-tuned encoders, despite the latter's strong in-domain performance.
• Limitations:
  • Input Conflation: The input representation ablation varied both field ordering and boundary tokens simultaneously, making it hard to isolate the exact cause of improvement.
  • Supervision Mismatch: Training used single aggregated labels, while testing used multi-annotator labels. The authors suggest future work could use soft labels or multi-label formulations to better capture annotation uncertainty.
  • Zero-Shot Scope: Decoder models were only evaluated zero-shot; parameter-efficient fine-tuning (e.g., LoRA) was not explored for these models.

In conclusion, the KCLarity team demonstrated that while RoBERTa-large is a strong baseline for political evasion detection, the superior generalization of GPT-5.2 on the hidden test set indicates a shift toward leveraging large, pre-trained decoder models for complex, subjective NLP tasks in political communication.