Clear Messages, Ambiguous Audiences: Measuring Interpretability in Political Communication

By analyzing confidence-weighted human annotations of political social media messages, this paper demonstrates that while political intent is generally clear, ambiguity is systematically concentrated in audience targeting, suggesting that measurement error in political text is driven by strategic communication rather than random noise.

The Gist

The Core Idea: The "Secret Handshake" Problem

Imagine you are watching a movie where a character walks into a room, looks directly at the camera, and says, "I am going to win this election!"

If you were asked to analyze that scene, you would find it very easy to answer two questions:

1. What is the character doing? (They are making a campaign promise.)
2. Who are they talking to? (The audience/the voters.)

In most movies, the "What" is crystal clear. But sometimes, the character says something that feels like a secret handshake. They might say something that sounds like a joke to their friends, but a serious promise to their supporters. You know what they said, but you’re scratching your head wondering, "Wait, who are they actually trying to impress right now?"

This paper is about that exact moment of confusion.

---

The Scientific Breakdown (The "Plain English" Version)

1. The Old Way of Thinking: "The Messy Room"

In the past, when researchers tried to use computers or humans to categorize political tweets or posts, they ran into disagreements. If one person thought a tweet was meant for "Young Voters" and another thought it was for "Old Voters," researchers usually just called that "noise"—like static on a radio or a messy room that needs cleaning. They assumed the disagreement was just human error.

2. What the Researchers Found: "The Clear Signal, The Blurry Target"

The researchers looked at 5,000 messages from U.S. politicians. They found that political messages are actually incredibly "legible" (easy to read). It’s not like a messy room; it’s more like a high-definition TV.

However, they discovered a specific pattern:

• The "What" is clear: When a politician speaks, we almost always know their intent. We know if they are attacking an opponent, praising a policy, or asking for money.
• The "Who" is blurry: The confusion happens when we try to guess the audience. Specifically, when politicians try to speak to their "constituency" (their specific group of voters), the messages become much harder to pin down.

3. The "Strategic" Twist: "The Double Agent"

This is the most important part. The researchers found that this confusion isn't just because humans are bad at guessing. It’s because politicians are being strategic.

Think of a politician like a Double Agent. They want to send a message that sounds one way to their "home team" (to get them excited) but sounds a completely different way to "the other side" (to avoid looking bad or to avoid being criticized).

Because the politician is intentionally being "vague" or "multi-purpose," the message becomes a puzzle. The uncertainty isn't a mistake in the measurement; the uncertainty is the point.

---

The Summary Metaphor: The "Swiss Army Knife" Message

If most political messages are like a hammer (one tool, one job, very clear), politicians often use Swiss Army Knife messages.

They craft a single sentence that acts as a hammer for their supporters, but also acts as a screwdriver for their opponents. When researchers try to label that message, they struggle—not because they are unobservant, but because the politician has intentionally built a tool that does two things at once.

The takeaway: When you see a politician being "vague," they probably aren't being clumsy; they are being tactical.

Technical Summary

Technical Summary: Clear Messages, Ambiguous Audiences: Measuring Interpretability in Political Communication

1. Problem Statement

In computational political science, text-based measurement (e.g., classifying a tweet as "inciting" or "policy-oriented") typically relies on human coders to create "ground truth" labels. A standard methodological assumption in the field is that disagreement between coders is stochastic noise—random error that can be smoothed out through aggregation or large sample sizes.

This paper challenges that assumption. It asks whether classification disagreement is truly random or if it is systematically structured by the nature of the political content itself. Specifically, the authors investigate whether certain dimensions of political communication (such as who a message is intended for) are inherently more ambiguous than others (such as what the message is about), and whether this ambiguity is a byproduct of strategic political communication.

2. Methodology

The researchers employed a high-granularity annotation framework to measure "interpretability" (the ease with which a message's meaning is understood).

• Dataset: 5,000 social media messages authored by U.S. politicians.
• Annotation Technique: Instead of simple binary labeling, the authors used confidence-weighted human annotations. Coders did not just label a message; they provided a confidence score indicating how certain they were about the classification. This allows for the mathematical separation of "true disagreement" from "uncertainty."
• Dimensions of Analysis: Messages were classified across several dimensions:
  • Message Type: The content/intent of the message.
  • Partisan Bias: The political leaning of the content.
  • Audience Classification: The intended target/constituency of the message.
• Statistical Controls: The authors utilized politician fixed effects to control for individual stylistic differences, ensuring that the observed patterns were properties of political communication at large rather than the idiosyncrasies of specific politicians.

3. Key Results

The study reveals a dichotomy between the intent of a message and its target:

• High Baseline Legibility: Overall, political communication is highly legible. The mean confidence score across all categories exceeds 0.99, suggesting that most political text is unambiguous.
• The "Constituency Penalty": While most dimensions show high agreement, a systematic pattern of uncertainty emerges in the audience/constituency category. This category exhibits a 1.79 percentage point penalty in confidence compared to other dimensions.
• Relative vs. Absolute Uncertainty: Although 1.79% sounds small, the authors note that because the baseline confidence is so high (0.99+), this penalty represents a sharp relative increase in interpretive uncertainty.
• Intent vs. Target: The findings show a decoupling of meaning: the intent of a message (what the politician is doing) is clear, but the audience targeting (who they are talking to) is intentionally or structurally ambiguous.

4. Key Contributions

• Methodological Critique: The paper provides empirical evidence that measurement error in political text is non-random and structured. It refutes the "noise" assumption, suggesting that disagreement in datasets often reflects meaningful ambiguity in the text.
• Theoretical Insight into Strategic Ambiguity: The results support the theory of strategic ambiguity. Politicians often craft messages that have a clear "core" (to satisfy their base) but an ambiguous "target" (to avoid alienating moderate voters or to allow for multiple interpretations).
• Refinement of Computational Metrics: By demonstrating that uncertainty is concentrated in the "audience" dimension, the paper suggests that researchers should be more cautious when using automated tools to classify political targeting compared to classifying political sentiment or topic.

5. Significance

This research has profound implications for both methodology and political theory.

For methodologists, it serves as a warning: if disagreement in your dataset is concentrated in specific dimensions (like audience), it is not "noise" to be ignored, but a signal of the text's inherent complexity. For political scientists, it provides a quantitative lens through which to study how politicians navigate the tension between being "clear" to their supporters and "vague" to their opponents, effectively using ambiguity as a tool of political survival.