💬 NLP

PERCEPT: A Corpus for POS Tagging and Analysis of Persian-English Code-Mixing

This paper introduces PERCEPT, the first large-scale Persian-English code-mixed corpus annotated with Universal Dependencies POS tags, along with an LLM-assisted annotation framework and a comprehensive linguistic analysis revealing platform-specific patterns in code-mixing across social media.

Original authors: Ghazal Kalhor, Zahra Jafari, Amirarsalan Shahbazi, Behnam Bahrak

Published 2026-08-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ghazal Kalhor, Zahra Jafari, Amirarsalan Shahbazi, Behnam Bahrak

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a giant, chaotic playground where people from all over the world are shouting, whispering, and chatting all at once. In this playground, many people don't just speak one language; they mix them together like ingredients in a smoothie. This is called "code-mixing." It's when someone might start a sentence in their native tongue and suddenly throw in an English word because it feels just right, or because it's the only word they know for a new gadget. For computers, this is a nightmare. If a computer is trying to understand a sentence, it expects words to follow strict rules. But when languages get mixed up, the computer gets confused, like a chef trying to follow a recipe that suddenly switches from metric cups to imperial ounces in the middle of baking a cake. To teach computers how to handle this messy, fun reality, scientists need a special dictionary and a set of rules—a "corpus"—that shows them exactly how these mixed words work.

Enter PERCEPT, a new project that acts like a giant, organized library for Persian speakers who love to mix English into their posts. The researchers, a team of linguists and computer scientists from Iran, noticed that while we have maps for other mixed-language pairs, the Persian-English mix was a dark, unexplored forest. They wanted to build a flashlight to see what was happening there. So, they collected 6,800 posts from three popular digital hangouts: X (formerly Twitter), Instagram, and Digikala (a massive online shopping site). They didn't just collect the words; they used a super-smart AI assistant named Gemini to act as a detective, tagging every single English word (or English word written in Persian letters) with its "job" in the sentence. In grammar, this job is called a "Part-of-Speech" (POS). Is the word a noun (a thing)? A verb (an action)? An adjective (a description)? The team also asked the AI to guess what the post was talking about, like "shopping," "politics," or "funny memes."

Here is the exciting part: the team didn't just build the library; they went inside and started reading the books to see what patterns they could find. They discovered that when Persian speakers mix in English, they mostly use it for nouns (things). It's like they are borrowing the "objects" from English but keeping the "actions" and "descriptions" in Persian. For example, they might say, "I bought a new laptop," where "laptop" is the borrowed noun, but the rest of the sentence stays Persian. However, the "flavor" of the mix changes depending on where you are. On the shopping site, Digikala, people mixed in a lot of brand names and product models (which are special names, or "proper nouns"). On X, they mixed in more verbs (actions), and on Instagram, they leaned toward adjectives (descriptions).

The researchers also looked at where these mixed words hide in a sentence. They found a surprising consistency: no matter which platform you use, mixed words tend to appear at the beginning or the middle of a sentence, rarely at the very end. It's as if the speakers drop the English word in early to set the scene, then finish the thought in Persian. But the most interesting discovery was about "triggering." On Digikala, if someone used one English word, they were very likely to use another one right after. It's like once you start talking about a specific brand or product, the English words just keep flowing. This didn't happen as much on Instagram or X.

Finally, they checked which topics caused the most mixing. Unsurprisingly, the topics of "Industry & Commerce" and "Brand & Business" had the highest amount of code-mixing. When people talk about shopping, tech, or business, they simply can't help but sprinkle in English terms. The team confirmed their AI's work by having human experts double-check a sample, and the humans agreed with the AI almost all the time, proving that their new library is reliable.

In short, this paper doesn't just give us a new dataset; it gives us a clear picture of how Persian speakers naturally blend languages. It suggests that while the types of words they borrow change based on the platform, the way they mix them follows a steady rhythm. This map is now open for anyone to use, helping to build better translation tools, smarter chatbots, and a deeper understanding of how we communicate in a world where languages are constantly dancing together.

Technical Summary: PERCEPT – A Corpus for POS Tagging and Analysis of Persian–English Code-Mixing

Problem Statement

Social media has become a primary venue for multilingual communication, where users frequently mix languages within single utterances, a phenomenon known as code-mixing. While code-mixed corpora exist for several language pairs (e.g., Malayalam–English, Hindi–English, Bengali–English), Persian–English code-mixing remains relatively underexplored. Existing Persian resources primarily focus on language identification or sentiment analysis and lack Universal Dependencies (UD) part-of-speech (POS) annotations specifically for code-mixed words. This absence limits both linguistic analyses and the development of syntax-aware Natural Language Processing (NLP) models capable of handling the complexities of Persian–English code-mixing, such as tokenization, language identification, and syntactic parsing.

Methodology

To address this gap, the authors introduce PERCEPT, the first publicly available large-scale Persian–English code-mixed corpus annotated with UD POS tags for code-mixed words and document-level topic labels.

Data Collection

The dataset comprises 6,800 texts collected from three major platforms: X (formerly Twitter), Instagram, and Digikala (an e-commerce platform). The authors aggregated data from existing Persian datasets and performed additional keyword-based searches on X to ensure coverage of frequent code-mixed words. The final dataset includes both English words written in the Latin script and English words transliterated into the Persian script. Offensive content was filtered out, and user mentions and URLs were anonymized.

Annotation Framework

The authors propose an LLM-assisted annotation framework utilizing Gemini 3.5 Flash to automate the identification and tagging process:

POS Tagging: The model identifies English and transliterated English words within Persian text and assigns one of 17 Universal Dependencies POS tags (e.g., NOUN, VERB, PROPN). The framework includes specific instructions to exclude well-established loanwords (e.g., "laptop," "internet") and platform-specific terms (e.g., "timeline" on X, "DigiPay" on Digikala) to focus on non-standard code-mixing.
Topic Detection: The model assigns document-level topic labels from a predefined taxonomy (e.g., Brand & Business, Industry & Commerce, Lifestyle) to each text.
Human Validation: To ensure reliability, two native Persian-speaking annotators with NLP coursework independently verified a random sample of 150 texts (268 code-mixed words). They evaluated both token identification and POS tagging without AI assistance.

Statistical Analysis

Using the annotated PERCEPT corpus, the authors conducted a comprehensive empirical analysis investigating four research questions:

Which POS categories are most likely to host code-mixing?
Where do code-mixed words tend to occur within an utterance (positional distribution)?
Does code-mixing exhibit a "triggering effect" (recurring within the same text)?
How does the degree of code-mixing vary across discussion topics?

Key Results

1. POS Distribution

Nouns are the predominant category for code-mixed words across all platforms, accounting for approximately 60% of instances on X and Instagram, and 45.4% on Digikala. Secondary distributions vary by platform:

X: Verbs are the second most frequent category (23.0%).
Instagram: Adjectives rank second (19.3%).
Digikala: Proper Nouns (PROPN) rank second (35.0%), attributed to the frequent use of English brand and product model names in e-commerce listings.

2. Positional Distribution

The positional distribution of code-mixed words is remarkably consistent across all three platforms. Approximately one-third of code-mixed words appear in the initial position, one-third in the medial position, and slightly fewer in the final position. Statistical testing (chi-square) found no significant association between platform and word position.

3. Triggering Effect

The "triggering effect" (the likelihood of multiple code-mixed words appearing in a single utterance) varies substantially by platform:

Digikala: Exhibits the highest triggering effect (42.0%), likely due to users discussing multiple product features, brands, or specifications in single reviews.
X: Ranks second (27.1%).
Instagram: Shows the lowest proportion (10.2%).

4. Topic Variation

Discussions related to Industry & Commerce and Brand & Business consistently exhibit the highest degree of code-mixing across all platforms. On Digikala, topic dependence is strongest, with commerce-related discussions containing significantly more code-mixed words than other topics. In contrast, Instagram shows a nearly constant degree of code-mixing across topics, suggesting users typically insert only a single English word regardless of the subject.

Annotation Reliability

Human evaluation confirmed the reliability of the LLM-generated annotations. For POS tagging, the agreement between human annotators yielded a Cohen's $\kappa$ of 0.894. When comparing the LLM's annotations against the gold standard, the model achieved an accuracy of 97.0% for POS tagging of correctly identified code-mixed tokens and a macro F1 score of 76.8%. For the complete annotation pipeline, including both token identification and POS assignment, the LLM achieved an overall F1 score of 88.1%.

Significance and Contributions

The paper claims three primary contributions:

Resource Creation: The introduction of PERCEPT, the first large-scale Persian–English code-mixed corpus annotated with UD POS tags and topic labels, filling a critical gap in low-resource NLP resources.
Methodological Framework: The proposal of an LLM-assisted annotation framework that demonstrates high reliability through human validation, offering a transferable approach for constructing linguistically annotated corpora for other low-resource languages.
Empirical Insights: The first large-scale empirical analysis of Persian–English code-mixing across multiple platforms, revealing that while nouns dominate code-mixing generally, secondary patterns and triggering effects are heavily influenced by the specific context and nature of the platform (e.g., e-commerce vs. social networking).

The authors position PERCEPT as a foundational resource for developing and evaluating POS taggers, language identification systems, and other syntax-aware NLP models for Persian. They also note that the dataset enables future linguistic studies on the structural and sociolinguistic properties of Persian–English code-mixing. The dataset and analysis code are publicly available to facilitate further research.