💬 NLP

IDIOLEX: Unified and Continuous Representations for Idiolectal and Stylistic Variation

This paper introduces IDIOLEX, a framework that learns continuous, content-decoupled representations of individual and community-level stylistic and dialectal variation by combining sentence provenance with linguistic features, demonstrating their effectiveness in dialect analysis and stylistic alignment for language models.

Original authors: Anjali Kantharuban, Aarohi Srivastava, Fahim Faisal, Orevaoghene Ahia, Antonios Anastasopoulos, David Chiang, Yulia Tsvetkov, Graham Neubig

Published 2026-04-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Anjali Kantharuban, Aarohi Srivastava, Fahim Faisal, Orevaoghene Ahia, Antonios Anastasopoulos, David Chiang, Yulia Tsvetkov, Graham Neubig

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Robot Voice"

Imagine you ask a robot for help. You say, "Hey, can you give me a hand with this? I'm totally lost, for real."

The robot replies, "Please formulate your inquiry in an orderly and exhaustive manner. Upon receipt of the complete exposition of facts, I shall proceed to emit a response..."

Even though the robot understood what you meant (the meaning), it completely missed how you said it (the style). It sounded like a stiff lawyer instead of a friendly friend.

Most AI models today are obsessed with meaning. They are like encyclopedias: they know facts, but they don't know how to sound like you. They struggle to understand that "What's up?" and "Greetings, how do you do?" mean the same thing but belong to totally different social worlds.

The Solution: IDIOLEX (The "Style Fingerprint")

The researchers created a new tool called IDIOLEX. Think of it as a Style Fingerprint Scanner.

Instead of asking, "What does this sentence say?", IDIOLEX asks, "Who wrote this, and what kind of vibe are they giving off?"

It creates a special map where sentences are placed based on their dialect (like a regional accent) and idiolect (your personal way of speaking).

The Goal: To separate the content (the story) from the style (the voice).
The Result: A sentence about "buying milk" spoken by a teenager in Buenos Aires and the same sentence spoken by a formal judge in Madrid end up in very different spots on the map, even though the meaning is identical.

How It Works: The "Detective Training"

How do you teach a computer to hear an accent without hiring 1,000 linguists to label every sentence? The authors used a clever trick called "Weak Supervision."

Imagine you are training a dog to recognize different breeds of dogs without showing it a textbook. Instead, you use proximity:

Same Comment: Two sentences from the same Reddit post? They are definitely from the same person. (High similarity).
Same Author: Two sentences from the same person, but different posts? They are very similar. (Medium similarity).
Same Community: Two sentences from people in the same city (e.g., both from Cairo)? They share a dialect. (Low similarity).
Different Community: A sentence from Cairo and one from Tokyo? Totally different. (No similarity).

The AI learns by looking at these relationships. It realizes, "Ah, sentences from the same person cluster together, and sentences from the same city cluster together, even if they are talking about different things."

They also used a "Smart Assistant" (an LLM) to act as a linguist, pointing out specific clues like "This sentence uses the word 'vos' instead of 'tú'" or "This sentence drops the 'g' sound." This helps the AI learn the specific rules of the dialect.

Why Is This Cool? (The Superpowers)

1. The "Style Translator"

Usually, if you want an AI to speak in a specific dialect, you have to write a huge manual of rules. With IDIOLEX, you just show the AI examples of that style.

Analogy: Imagine you want to learn to speak like a surfer. Instead of reading a dictionary of slang, you just hang out with surfers. IDIOLEX lets the AI "hang out" with millions of Reddit comments to learn the vibe naturally.

2. The "Identity Detective"

The paper tested IDIOLEX on two hard tasks:

Dialect Identification: Guessing where someone is from just by their text. IDIOLEX was better than previous models at guessing if someone was from Argentina vs. Spain, or Egypt vs. Morocco.
Authorship Attribution: Guessing who wrote a text. It's like a forensic linguist. If a mystery novel has a suspicious note, IDIOLEX can tell you if it was written by the same person who wrote the diary entries, even if the topics are totally different.

3. The "Tuning Knob" for AI

The most exciting part is using IDIOLEX to fix other AIs.
Currently, if you ask an AI to speak in a specific dialect, it often sounds fake or breaks its grammar. The researchers used IDIOLEX as a "training objective."

The Metaphor: Imagine you are teaching a student to write like a poet. Instead of just grading them on spelling, you give them a "Poetry Score" based on how much their writing feels like a poem. IDIOLEX gives the AI a "Dialect Score." The AI tries to maximize this score, learning to sound authentic without losing its ability to speak clearly.

The Bottom Line

IDIOLEX is a framework that teaches computers to listen to the music of language, not just the lyrics.

By mapping out how different people and communities speak, it helps us build AI that doesn't just sound like a robot, but can actually adapt to sound like a friend, a local, or even you. This makes AI more inclusive, more useful for diverse communities, and much less likely to sound like a stiff bureaucrat.

1. Problem Statement

Current Large Language Model (LLM) evaluation and optimization heavily prioritize semantic correctness (what a sentence says) while neglecting stylistic and dialectal variation (how a sentence is expressed). This gap leads to several issues:

Misalignment: LLMs often fail to adapt to specific user idiolects (individual language habits), registers, or dialects, causing misunderstandings.
Task Isolation: Prior work treats dialect identification (DID) and authorship attribution (AA) as separate, discrete classification tasks, failing to capture the continuous nature of stylistic similarity.
Semantic Interference: Existing sentence embeddings are optimized for semantic similarity, making it difficult to isolate stylistic features from topical content.

The authors propose Idiolectal Representation Learning (IRL): learning sentence representations that capture style and dialect while decoupling them from semantic content.

2. Methodology: The IDIOLEX Framework

IDIOLEX is a training framework designed to learn continuous sentence representations that encode idiolectal variation using minimal supervision. It combines weak supervision from proximity hierarchies with LLM-extracted linguistic features.

A. Data and Supervision Signals

The framework utilizes publicly available Reddit comments from regional communities (Arabic and Spanish). Instead of relying on hard labels, it uses a Proximity Hierarchy as weak supervision:

Same Comment ( $r=3$ ): Highest stylistic similarity.
Same Author ( $r=2$ ): High similarity.
Same Dialect Community ( $r=1$ ): Moderate similarity.
Different Community ( $r=0$ ): Lowest similarity.

Additionally, to mitigate noise in the proximity labels, the framework incorporates LLM-extracted linguistic features. Using an LLM (GPT-5 mini), the system extracts binary indicators of specific dialectal markers (e.g., specific morphological suffixes, negation particles, orthographic variations) based on linguistic textbooks.

B. Architecture and Training

IDIOLEX builds upon a pre-trained sentence encoder (e.g., AraBERT v2 for Arabic, BERTIN for Spanish) with a two-stage training process:

Stage 1: Margin Ranking Pre-training
- The model is trained on the full dataset using a Margin Ranking Loss (MRL).
- Objective: Enforce that sentences with higher proximity scores (e.g., same author) have higher cosine similarity in the embedding space than those with lower scores.
- This stage shifts the model's focus from semantic to stylistic encoding using large amounts of unannotated data.
Stage 2: Feature-Aware Fine-tuning
- Trained on a smaller subset with linguistic feature annotations.
- Auxiliary Heads:
  - Feature Prediction Head: A binary cross-entropy loss to predict the extracted linguistic features.
  - Supervised Contrastive Loss (SCL): Uses Jaccard similarity of feature vectors to weight the contrastive loss, encouraging the model to group sentences with similar linguistic structures.
- Regularization: Includes Centering & Variance Constraints (VICReg-style) to prevent representation collapse and ensure a well-conditioned, isotropic embedding space.

The final loss function combines the ranking loss, feature prediction loss, and supervised contrastive loss.

3. Key Contributions

Unified Framework: IDIOLEX is the first framework to jointly model individual (authorship) and community-level (dialect) variation in a single continuous representation space, moving beyond discrete classification.
Semantic Decoupling: The learned representations explicitly prioritize linguistic form over topical content, achieving low correlation with semantic similarity scores (Pearson $r \approx 0.09$ for Spanish, $0.19$ for Arabic).
Language Agnosticism: The approach is adaptable to any language by swapping the underlying pre-trained encoder and linguistic feature definitions.
Post-Training Alignment Objective: The authors demonstrate that IDIOLEX embeddings can serve as a training objective for Supervised Fine-Tuning (SFT) of LLMs to improve dialectal alignment without sacrificing fluency.

4. Experimental Results

The authors evaluated IDIOLEX on Arabic and Spanish datasets across three contexts:

A. Downstream Classification Tasks

Dialect Identification (DID): On the DSL-ML (Spanish) and MADAR 26 (Arabic) benchmarks, IDIOLEX outperformed previous neural baselines and state-of-the-art shared-task submissions.
- Spanish (DSL-ML): F1 Score of 0.85 (vs. 0.82 for the previous best).
- Arabic (MADAR 26): Accuracy of 0.66 (near the best reported result of 0.67, achieved without task-specific heuristics).
Authorship Attribution (AA): On the PAN 19 open-set benchmark, IDIOLEX achieved 38% accuracy (Fine-tuned + Lexical), significantly outperforming the 28% achieved by a standard fine-tuned BERT baseline. This demonstrates strong cross-domain generalization.

B. Disentanglement Analysis

Semantic vs. Stylistic: As shown in Figure 4, there is a weak positive correlation between IDIOLEX similarity and semantic similarity (Multilingual-E5), confirming that the model successfully isolates stylistic signals.
Geographic Gradient: On parallel sentences (MADAR 26) with identical semantics, IDIOLEX correctly ranked stylistic similarity based on geographic proximity (Same Country > Same Region > Different Region), proving it captures fine-grained dialectal structure independent of meaning.

C. LLM Post-Training (Style Alignment)

The authors fine-tuned Llama 3.1 and Allam (Arabic LLM) using IDIOLEX embeddings as an alignment objective.
Results: The IDIOLEX-aligned models achieved superior Dialectal Adherence (ADI2) scores compared to standard SFT, while maintaining or improving translation quality (ChrF++).
Efficiency: The models achieved strong performance on low-resource dialects (e.g., Palestinian Arabic) with only ~50k training pairs, outperforming models trained on millions of pairs that lacked this specific stylistic objective.

5. Significance and Impact

Paradigm Shift: IDIOLEX advocates for treating idiolect as a graded, continuous phenomenon rather than a set of discrete tags. This allows for more nuanced analysis of language variation.
Inclusive AI: By enabling models to better understand and generate diverse dialects and styles, IDIOLEX supports the development of more accessible and culturally inclusive LLMs.
Practical Utility: The framework provides a reusable, language-agnostic tool for tasks ranging from forensic linguistics (authorship attribution) to improving the "human-like" adaptability of chatbots.
Ethical Awareness: The paper acknowledges the risks of dialectal modeling (e.g., stereotyping or inauthentic generation) and emphasizes the need for user control and contextual sensitivity in deployment.

In conclusion, IDIOLEX successfully bridges the gap between semantic understanding and stylistic variation, providing a robust foundation for next-generation language models that are sensitive to the rich diversity of human language.