💬 NLP

HebID: Detecting Social Identities in Hebrew-language Political Text

This paper introduces HebID, the first multilabel Hebrew corpus for detecting nuanced social identities in political discourse, which is used to benchmark language models and analyze the alignment between elite identity expression and public priorities in Israel.

Original authors: Guy Mor-Lan, Naama Rivlin-Angert, Yael R. Kaplan, Tamir Sheafer, Shaul R. Shenhav

Published 2026-02-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Guy Mor-Lan, Naama Rivlin-Angert, Yael R. Kaplan, Tamir Sheafer, Shaul R. Shenhav

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking through a bustling marketplace where politicians are shouting their messages to the crowd. They talk about everything from the economy to religion, but they often do it by wearing "invisible badges" that signal who they are and who they are talking to. Some wear a badge that says "I'm a traditionalist," others wear "I'm a modern liberal," and some wear "I'm a security hawk."

For a long time, computers (AI) have been great at reading English texts and spotting these badges. But when it comes to Hebrew, the language of Israel, the computers were mostly blind. They didn't have a dictionary or a rulebook to understand these subtle social identities in Hebrew political speech.

Enter HEBID. Think of HEBID as the first-ever giant, high-tech translator's guidebook specifically designed to spot these social identity badges in Hebrew political text.

Here is a breakdown of how they built it and what they found, using simple analogies:

1. Building the Rulebook (The Dataset)

The researchers didn't just guess which badges mattered. They went to the source: the people.

The Survey: They asked thousands of regular Israeli citizens, "What groups do you feel you belong to?" (Like: "I'm a right-winger," "I'm religious," "I care about social justice").
The Selection: Based on what people actually said mattered to them, the researchers picked the top 12 most popular "badges."
The Training: They took 5,500 sentences from Israeli politicians' Facebook posts and had human experts manually tag every sentence with the correct badges. For example, a sentence might wear two badges at once: "Rightist" and "Security-oriented."
The Result: A massive, labeled library of Hebrew sentences that teaches computers what these specific identities look like in real life.

2. The Race to Find the Best Detective (The Models)

Once they had the training data, they needed to teach a computer to spot these badges automatically. They ran a "detective race" with three types of AI:

The Old School Detectives (Encoders): These are standard AI models that read text and classify it.
The New Super Detectives (Decoder LLMs): These are the latest, massive AI models (like the ones powering chatbots) that can generate text.
The Winner: The Hebrew-tuned Super Detective (specifically a model called DICTALM2.0) won the race. It was like giving the detective a native Hebrew speaker's intuition. It understood the cultural nuances and political slang better than the older models, achieving a success rate of about 74% (which is very high for this complex task).

3. Putting the Detective to Work (The Findings)

Once the AI was trained, the researchers used it to analyze three different "rooms" in the political house:

Facebook: Where politicians talk to voters directly.
The Knesset (Parliament): Where they give formal speeches to each other.
The Public: What regular people say in surveys.

Here are the cool things they discovered:

The "Election Bump": Just like a store puts up "SALE" signs before a holiday, politicians suddenly start wearing more identity badges right before elections. The AI spotted that identity talk spikes dramatically during election seasons.
The "Bundle" Effect: Some badges always come together. If a politician wears the "Rightist" badge, they are very likely to also be wearing the "Security" and "Zionist" badges. It's like a uniform; you don't see the "Leftist" badge paired with the "Ultra-Orthodox" badge very often.
The Gender Gap: The AI noticed that men and women politicians wear different badges. Men tended to talk more about "Security" and "Capitalism," while women were much more likely to wear the "Socially-Oriented" (caring about welfare and community) badge.
The "Two Worlds" Problem: Sometimes what politicians say and what the public feels don't match. For example, the public cares a lot about "Honesty" and "Democracy," but politicians on Facebook talk about those less than they talk about "Rightist" or "Security" issues. However, in the Parliament, politicians talk more about social issues than they do on Facebook.

Why Does This Matter?

Think of HEBID as a new pair of glasses for social scientists.

Before: They could only see the "big picture" (e.g., "This party is Left or Right").
Now: They can see the "fine details" (e.g., "This politician is trying to appeal to religious voters and security voters simultaneously").

This tool isn't just for Hebrew. It shows researchers how to build similar tools for other languages that aren't English, helping us understand how people in different cultures use language to signal who they are and who they want to vote for.

In short: HEBID is a smart, culturally-aware AI that finally learned how to read the "invisible badges" of Israeli politics, revealing how politicians try to connect with different groups of people in ways we couldn't measure before.

1. Problem Statement

Social identities (e.g., political ideology, religious affiliation, demographic groups) are critical drivers of political behavior and discourse. However, existing Natural Language Processing (NLP) resources for identity detection suffer from three major limitations:

Language Bias: They are predominantly English-centric, lacking resources for low-resource languages like Hebrew.
Granularity: They rely on coarse-grained categories (e.g., party or ethnicity) rather than nuanced social identities.
Labeling Capacity: They are typically single-label, failing to capture the reality that political text often expresses multiple, overlapping identities simultaneously.

The authors aim to bridge this gap by introducing HEBID, the first publicly available, fine-grained, multilabel Hebrew corpus for social identity detection in political text.

2. Methodology

A. Dataset Construction (HEBID)

Source Data: 5,536 sentences sampled from 4,800 unique Facebook posts by Israeli politicians, parties, and candidates (Dec 2018 – Apr 2021). This period covers four rapid-fire Israeli elections.
Identity Selection: Instead of arbitrary selection, the 12 target identities were derived from a 12-wave national panel survey ( $N=1,769$ $N = 1, 769$ ). The survey asked respondents to select up to three identities from a list of 28. The 12 identities included in HEBID were those that consistently surpassed a 5% selection threshold in the first five waves.
- The 12 Identities: Rightist, Leftist, Conservative, Liberal, Socially-oriented, Capitalist, Zionist, Palestinian, Honest, Security-oriented, Ultra-Orthodox, Democrat.
Annotation Scheme:
- Multilabel: Each sentence is annotated for all 12 identities simultaneously.
- Positivity Constraint: Identities are only annotated if referenced positively; negative or oppositional mentions are excluded.
- Reliability: Two authors annotated the data. Inter-annotator agreement (Cohen's $\kappa$ ) averaged 0.77, with a mean percent agreement of 96.1%.
Data Split: 70% Training, 15% Validation, 15% Test.

B. Modeling and Experiments

The authors benchmarked three categories of models on the Facebook data:

Baseline Classifiers: Logistic Regression and LinearSVC using TF-IDF features.
Encoder Models: Fine-tuned multilabel and single-label versions of:
- Multilingual BERT (mBERT)
- Hebrew-specific encoders: ALEPHBERT, HERO, DICTABERT (Base and Large).
Decoder LLMs: Fine-tuned (via QLORA) 2B–9B parameter models:
- GEMMA 2 (2B, 9B)
- QWEN3-8B
- DICTALM2.0 (A Hebrew-tuned model based on Mistral 7B, pre-trained on 100B Hebrew tokens).

The task was framed as sentence-level classification (encoders) or generation of a comma-separated list of labels (decoders).

C. Validation and Generalization

Cross-Genre: The best model was applied to 500 parliamentary speech excerpts (Knesset) from the same time period to test generalization.
External Validity: The classifier's output was correlated with the CHES-Israel expert survey (party policy rankings) to ensure the detected identities align with known political stances.
Triangulation Analysis: The authors compared identity prevalence across three sources: Facebook posts, Knesset speeches, and the public panel survey.

3. Key Results

A. Model Performance

Best Performer: DICTALM2.0 (Hebrew-tuned decoder) achieved the highest Macro-F1 score of 0.743.
Comparison: It outperformed the best encoder-only baseline (DICTABERTLarge) by over 6 points (0.678 F1).
Insight: The superior performance of the Hebrew-tuned decoder highlights the necessity of language-specific cultural and political context for identity detection.
Multilabel vs. Single-Label: Multilabel encoder models outperformed the average of single-label models, indicating that joint learning captures the interrelated nature of identities.

B. Generalization

The DICTALM2.0 model generalized well to parliamentary speeches, achieving a Macro-F1 of 0.72, confirming robustness across genres (social media vs. formal speech).

C. External Validity

Strong correlations were found between the model's predicted identity discourse and expert-coded party policy positions (CHES-Israel).
13 out of 21 correlations were statistically significant ( $p \le 0.05$ ) with strong effect sizes ( $|r| = 0.71$ to $0.94$).

D. Sociolinguistic Findings

Using the classifier, the authors uncovered several dynamics:

Popularity: "Socially-oriented," "Rightist," and "Democrat" were the most prevalent identities across sources. "Leftist" was notably less popular.
Temporal Trends: Identity discourse peaks significantly around election dates.
Elite vs. Public Gap:
- "Socially-oriented" identity surged in parliament after the March 2020 election but declined in public survey responses.
- "Honest" and "Democrat" became more popular with the public post-election but less so in elite discourse.
Identity Bundles: Factor analysis revealed two primary dimensions: a Left-wing group (Leftist, Democrat, Honest, Liberal, Palestinian) and a Right-wing group (Rightist, Conservative, Zionist, Security-oriented, Capitalist, Ultra-Orthodox).
Gender Differences: Women expressed more identities on average than men. "Socially-oriented" leaned significantly toward women, while "Rightist," "Security-oriented," and "Capitalist" leaned toward men.

4. Key Contributions

Dataset & Methodology: Introduction of HEBID, the first Hebrew multilabel corpus for social identity, grounded in empirical survey data rather than expert intuition alone.
Benchmarks: Comprehensive evaluation of modern NLP architectures, establishing that Hebrew-tuned decoder LLMs are the state-of-the-art for this task.
Cross-Genre Robustness: Demonstration that models trained on social media generalize effectively to formal parliamentary speech.
External Validation: Proof that computational identity detection aligns with expert political science data (CHES).
Sociolinguistic Insights: Novel findings on how identity expression varies by platform, time (election cycles), and gender in the Israeli context.

5. Significance

For Computational Social Science: Provides a robust tool to analyze political discourse in Hebrew, enabling the study of identity dynamics in a non-English, high-conflict political environment.
For NLP: Establishes a challenging new benchmark for multilabel classification in low-resource languages and demonstrates the efficacy of culturally-tuned LLMs over generic multilingual models.
Methodological Template: Offers a replicable framework for creating culturally-aware identity datasets in other non-English contexts by linking large-scale survey data to text annotation.

6. Limitations

Scope: Data is limited to Facebook and a specific time window (2018–2021); other platforms (Twitter, news) are excluded.
Population: The underlying survey only sampled Jewish citizens, excluding Palestinian citizens of Israel.
Annotation: Only positive identity expressions are captured; negative/critical references are excluded.
Bias: Models may inherit biases from pre-training data and may struggle with dialectal or informal text.