💻 computer science

SLAyiNG: A Diverse and Community-validated Dataset of Queer Slang

This paper introduces "SLAyiNG," the first community-validated dataset of over 500 English queer slang terms across 20 subcommunities, which reveals that while many language models lack representation bias against queer identities, they still struggle to process queer vernacular—particularly slang from African-American and Latine communities—highlighting the need for diverse linguistic resources to improve NLP systems.

Original authors: Leonor Veloso, Lea Hirlimann, Lucija Mihić Zidar, Philipp Wicke, Valentin Hofmann, Hinrich Schütze

Published 2026-08-14

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Leonor Veloso, Lea Hirlimann, Lucija Mihić Zidar, Philipp Wicke, Valentin Hofmann, Hinrich Schütze

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where computers are like giant, voracious readers that have swallowed almost every book, website, and conversation ever written. These machines, known as Large Language Models (LLMs), are the brains behind many of the chatbots and AI tools we use today. They learn by spotting patterns in words, trying to guess what comes next in a sentence. But here's the catch: if a computer has never heard a specific way of speaking, it gets confused. It might think a friendly joke is an insult, or it might fail to understand a cultural reference entirely. This is a big problem for "sociolects"—specialized languages used by specific groups of people, like teenagers, gamers, or the LGBTQ+ community. When AI doesn't understand these groups, it can accidentally be mean, misidentify people, or just fail to chat properly. The question researchers are asking is: How do we teach these digital brains to understand the rich, colorful, and evolving slang of queer communities without getting it wrong?

This is where a new project called SLAYING comes in. Think of it as a massive, community-approved dictionary and storybook specifically for queer slang. The researchers realized that while AI is getting better at understanding many types of language, it often stumbles over queer vernacular. Sometimes, the AI gets so confused that it mistakes a harmless, celebratory phrase for hate speech, or it generates a rude response to a user just because they used a specific word. To fix this, the team built the first real-world dataset of over 500 queer slang terms, complete with 3,405 examples of how these words are actually used in sentences. They didn't just pull these words from a textbook; they gathered them from movies, podcasts, and social media, and then had real members of the queer community check every single example to make sure it was used correctly and wasn't harmful.

The team used this new dataset to test some popular AI models, and they found something surprising. They discovered that a computer can be "nice" to queer people (meaning it doesn't hold stereotypes or hate them) but still be "clueless" about their language. It's like having a friend who loves you but doesn't understand your favorite inside jokes; they aren't being mean, they just don't get the reference. The study showed that when they taught an AI model using this new dataset of harmless, community-approved slang, the model actually got better at understanding queer people in general, not just the words.

However, the researchers also found that the AI isn't treating everyone equally. The models struggled the most with slang from specific groups, particularly those from African-American and Latine queer communities, as well as terms related to drag and non-binary identities. It's as if the AI has a blind spot for the very communities that created much of the slang it's trying to learn. Furthermore, the study showed that automated systems designed to catch "toxic" language often flag these queer terms as dangerous, even when they are being used in a positive, reclaimed way. This suggests that the very tools meant to keep the internet safe might be accidentally censoring queer voices. The paper suggests that by feeding these models more diverse, real-world examples of queer language, we can help them understand the community better, reducing both their confusion and their tendency to wrongly flag innocent speech as harmful.

Technical Summary: SLAyiNG – A Diverse and Community-validated Dataset of Queer Slang

Problem Statement

Despite advancements in Natural Language Processing (NLP) for other sociolects and informal languages, queer vernacular remains significantly underrepresented in evaluation benchmarks and resources. Current NLP systems frequently process queer language incorrectly, leading to misclassification as hate speech, the generation of negative responses, misgendering, and stereotyping. Existing literature on queer language in NLP is limited, often focusing on single features (such as reclaimed slurs) or utilizing small, synthetic lists of terminology. There is a critical lack of diverse, non-synthetic, and community-validated resources necessary to evaluate and mitigate anti-queer linguistic biases in AI systems.

Methodology

The authors introduce SLAyiNG, the first real-world dataset of English queer slang, constructed through a rigorous, multi-stage curation and annotation pipeline.

Data Collection and Sourcing

Terminology Sourcing: The authors aggregated slang terms from four primary sources: the Gender, Sex, and Sexual Orientation Ontology (GSSO), lgbtDB, the Chew Inclusive Terminology Glossary, and Wiktionary. This yielded an initial pool of 695 unique terms and 90 variants.
Raw Data Acquisition: To capture real-world usage, the team scraped 197,958 sentence-level examples from three sources: OpenSubtitles (movie/TV subtitles), Reddit (forum posts), and Podscripts (podcast transcripts).
Pre-filtering: Due to the ambiguity of many slang terms (e.g., "mother" or "read" having non-queer meanings), the authors employed GPT-5 Mini as a pre-filtering model for sense disambiguation. This model was validated against a gold standard set annotated by three authors, achieving a micro F1 score of 0.78.

Annotation Pipeline

The dataset underwent a three-task annotation process involving internal authors and a crowdsourced cohort of 33 diverse volunteers:

Sense Disambiguation: Determining if a specific definition applies to the term's usage in a given sentence.
Harmful Speech Detection: Identifying whether a sentence is harmful, distinguishing between derogatory usage and reclaimed/ingroup usage.
Author Identity Identification: Determining if the author of the sentence belongs to the "ingroup" (the community referenced by the term) or the "outgroup," as this heavily influences the perceived harm of slurs.

Validation and Release:

Internal Annotation: 5,973 examples were manually annotated. A "sanity check" batch ensured guideline stability.
Crowdsourced Annotation: 12% of the dataset was annotated by community volunteers to validate correctness and safety.
Dataset Versions: The final public version contains 3,405 entries labeled as harmless. A private version containing 121 additional entries (labeled as "uncertainly harmful" but confirmed as ingroup speech) is available upon request to prevent misuse while preserving linguistic nuance.

Key Contributions

SLAyiNG Dataset: A manually annotated, non-synthetic dataset containing over 500 queer slang terms used in 3,405 sentence contexts, representing more than 20 queer subcommunities (including drag, non-binary, African-American, and Latine communities).
Decoupling Representation and Linguistic Bias: The authors demonstrate that a language model can be unbiased regarding representation (stereotyping identities) while still exhibiting linguistic bias (poor performance on the community's language). They show a negative correlation between a model's performance on queer slang (measured by Bits Per Byte) and its bias score on the WINOQUEER benchmark.
Intersectional Bias Analysis: The study reveals that NLP models perform significantly worse on slang pertaining to specific subcommunities, particularly non-binary and drag communities. Furthermore, models exhibit higher error rates and toxicity classification false positives for slang associated with African-American and Latine communities compared to non-ethnicity-specific terms.

Experimental Results

The authors utilized SLAyiNG to conduct two case studies using the DATADECIDE 1B model suite and various data recipes:

Linguistic vs. Representational Bias: Analysis of 25 different data recipes revealed a negative correlation ( $r = -0.34$ , $p = 0.09$ ) between a model's fit to queer slang (lower BPB) and its anti-queer bias score (lower is better). While this correlation was not statistically significant at the 0.05 level, the authors hypothesize that it may indicate that corpora containing queer content can inadvertently reinforce queerphobic stereotypes (either through harmful content or outgroup authorship), suggesting that representation and linguistic bias may be partially decoupled.
Mitigation via Continued Pretraining: The authors fine-tuned an OLMo-1B model on SLAyiNG. This resulted in a reduction of the WinoQueer bias score across all community subsets and a decrease in BPB (improved language fit), suggesting that exposing language models to harmless, ingroup examples of queer language has the potential to mitigate both representational and linguistic biases.
Subcommunity Performance Disparities: Performance metrics (BPB) were consistently worse for text containing non-binary and drag slang. Similarly, slang terms associated with Black, African-American, and Latine communities yielded higher BPB (worse performance) than terms without specified ethnicity.
Toxicity Classification Errors: Hate speech classifiers (DOLMA and TOXIGEN's HateBERT) incorrectly labeled a significant portion of SLAyiNG examples as toxic (7.81% and 31.16%, respectively). This false positive rate was particularly high for drag slang and terms with semantic extensions (e.g., "cunty"), which classifiers often misinterpret through their mainstream, misogynistic definitions rather than their reclaimed queer meanings.

Significance and Claims

The paper argues that SLAyiNG provides a necessary foundation for improving the fairness and utility of NLP systems for queer stakeholders. The authors claim that:

Ecological Validity: Unlike synthetic datasets, SLAyiNG offers a realistic testbed for analyzing how linguistic biases affect queer users, moving beyond single-feature analysis to intersectional evaluation.
Bias Nuance: The findings challenge the assumption that unbiased models are inherently capable of processing diverse language; representation bias and linguistic bias are distinct phenomena that require different mitigation strategies.
Community-Centric Development: By involving the queer community in the annotation and validation process, the dataset ensures that the definitions of harm and identity are grounded in the lived experiences of the stakeholders, rather than external assumptions.

The authors conclude that while SLAyiNG is a significant step forward, it serves as a snapshot of currently available resources. They emphasize the need for ongoing revisions and extensions to keep pace with the evolving nature of queer vernacular and to address the limitations of current small-scale models in extrapolating to larger AI systems.