💬 NLP

The Algorithmic Unconscious: Structural Mechanisms and Implicit Biases in Large Language Models

This paper introduces the concept of the "algorithmic unconscious" to argue that significant biases in large language models arise not merely from dataset composition but from inherent structural mechanisms like tokenization, attention, and alignment, demonstrating through empirical analysis of Arabic language over-segmentation how these infrastructural factors systematically distort model performance and necessitate a new framework for technical auditing.

Original authors: Philippe Boisnard

Published 2026-02-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Philippe Boisnard

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The AI's "Hidden Brain"

Imagine you are talking to a very smart, well-read robot. You ask it a question, and it gives you an answer. But what if the robot has a "hidden brain" that it doesn't know about, and you can't see?

The author, Philippe Boisnard, calls this the "Algorithmic Unconscious."

Usually, when we think AI is biased, we blame the people who built it or the books they fed it. But this paper argues that the bias comes from the machine's own internal plumbing. Even if the builders are nice and the books are fair, the way the machine thinks (its math and structure) creates hidden prejudices. The machine doesn't "know" it's being unfair; it just happens automatically.

Here are the four main ways this "hidden brain" causes trouble, explained with analogies:

1. The "Pixelated Map" (Tokenization Bias)

The Problem: To understand you, an AI breaks your words into tiny chunks called "tokens." Think of these like puzzle pieces.
The Analogy: Imagine you have a map of the world.

English is drawn on the map with big, clear, easy-to-read blocks.
Arabic or Darija (North African dialects) are drawn with tiny, microscopic dots.

Because the AI's "puzzle pieces" are designed for English, when you type in Arabic, the AI has to chop your words into way more tiny pieces to understand them.

The Result: It takes the AI four times longer to "read" an Arabic sentence than an English one. It costs more money to run, and the AI gets "tired" (runs out of memory) faster. It's like trying to read a book where every word is written in a font so small you need a microscope, while everyone else reads in normal size.

2. The "Gossip Machine" (Causal Bias)

The Problem: AI doesn't understand why things happen; it only knows what usually happens next.
The Analogy: Imagine a gossip machine that has read every newspaper in the world.

If the machine sees "Fire" and "Smoke" together a million times, it learns they go together.
But if the machine sees "Islam" and "Violence" together in 10,000 news articles (even if it's rare in real life), it starts thinking they are causes of each other.

The AI can't tell the difference between a correlation (things happening together) and a cause (one thing making the other happen). Because it relies on statistics, it reinforces stereotypes. If the data says "X is often linked to Y," the AI assumes X causes Y, even if that's not true. It creates a "causal reductionism," flattening complex cultures into simple, often negative, stereotypes.

3. The "Melting Pot" (Dimensional Collapse)

The Problem: AI tries to be efficient. It loves patterns that happen often and ignores the weird, rare stuff.
The Analogy: Imagine a giant, magical smoothie blender.

You throw in a cup of English words (the most common fruit).
You throw in a cup of Darija (a rare fruit).
The blender spins. Because there is so much English, the Darija gets crushed and mixed in until it looks exactly like the English. The unique flavor of the Darija disappears.

In the AI's "mind" (its mathematical space), rare dialects get squashed. The AI forces Moroccan Darija to look like standard Arabic, and standard Arabic to look like English. It erases the unique identity of the language because it's "statistically safer" to just use the dominant version. This is a form of digital erasure—your culture isn't banned; it's just made invisible by the math.

4. The "Strict Librarian" (Safety Alignment)

The Problem: To stop AI from saying mean things, humans program it with "safety rules."
The Analogy: Imagine a strict librarian who only knows the rules of New York City.

If you speak in a New York slang, the librarian understands you perfectly.
If you speak in a Moroccan dialect, the librarian doesn't understand the context. They hear a word that sounds like an insult in New York, so they kick you out of the library.

The AI's "safety filters" are trained mostly on Western, English-speaking norms. They don't understand that a word might be a friendly joke in one culture but an insult in another. So, they accidentally silence minority voices, flagging normal conversation as "toxic" and refusing to answer questions that are perfectly fine in their own culture.

The Solution: A "Technical Clinic"

The author says we can't just "fix" the AI like a broken toaster. We need a Technical Clinic.

Don't blame the robot: The robot isn't "evil"; it's just following its hidden programming.
Diagnose the symptoms: We need to act like doctors. We need to measure exactly how many "puzzle pieces" Arabic needs compared to English. We need to map where the AI's "mind" is squashing certain cultures.
Prescribe a cure:
- Better Tools: Build puzzle pieces that work for everyone, not just English speakers.
- Better Data: Feed the AI more diverse stories, not just the "clean" ones from big newspapers.
- Better Rules: Teach the AI that "safety" looks different in different cultures.

The Takeaway

The "Algorithmic Unconscious" is the invisible set of rules that decides who gets heard and who gets silenced in the digital world. It's not a conspiracy; it's a structural flaw.

If we don't look under the hood and understand these hidden mechanisms, we risk building a future where AI helps everyone except the people who speak differently, think differently, or live in different cultures. We need to make the invisible visible so we can fix it.

1. Problem Statement

The paper addresses the limitation of current AI bias research, which predominantly attributes bias to dataset composition (e.g., underrepresentation in training data) or human intentionality (developer prejudice). The author argues that this perspective overlooks a deeper layer of bias: the Algorithmic Unconscious.

This concept refers to structural determinations inherent to the technical architecture of Large Language Models (LLMs) that operate without the model's reflexivity or the user's awareness. These biases emerge directly from mechanical processes—tokenization, attention mechanisms, statistical optimization, and safety alignment—rather than solely from the data or the creators. The paper posits that these mechanisms systematically reproduce and amplify linguistic and cultural hierarchies, particularly against low-resource languages like Arabic and its dialects (e.g., Darija), creating a form of "computational colonialism."

2. Methodology

The study employs a mixed-method approach combining theoretical framework analysis with empirical technical auditing:

Conceptual Framework: The author adapts the psychoanalytic concept of the "unconscious" to AI, defining it structurally as a set of determinations that act without being "thought" by the system and escape the system's own reflexivity. This distinguishes the work from previous uses of the term (e.g., Possati), which focused on the projection of the human unconscious onto machines.
Comparative Tokenization Analysis:
- Corpus: A set of 10 parallel sentences translated into English, Modern Standard Arabic (MSA), and Maghrebi Darija (both in Arabic script and Arabizi/Latin script).
- Infrastructures Tested: OpenAI (GPT-4.1 using tiktoken), Mistral-7B (using SentencePiece), and Anthropic.
- Metric: Token count inflation ratios relative to English.
Theoretical Synthesis: The paper integrates findings from latent space geometry (dimensional collapse), causal inference theory (Pearl), and critical data studies (Spivak, Couldry & Mejias) to map four distinct structural mechanisms of bias.

3. Key Contributions

The paper introduces the Algorithmic Unconscious as a critical lens for understanding AI bias, identifying four specific structural mechanisms:

Tokenization as Infrastructural Bias: Tokenizers do not treat all languages equally. They create an "internal geography" where English is central and efficient, while other languages are fragmented.
Causal Bias (Correlation $\neq$ Causation): LLMs learn conditional distributions, not causal mechanisms. This leads to the essentialization of underrepresented cultures, where complex cultural contexts are reduced to statistical correlations found in majority-language datasets.
Dimensional Collapse (Erasure): Due to optimization logic (minimizing loss), models favor frequent tokens. Minority linguistic features are mathematically "collapsed" into dominant attractors (e.g., Darija embeddings converging toward MSA), rendering dialect-specific nuances illegible.
Normative Bias in Safety Alignment: Safety filters (RLHF, guardrails) impose a "Western-centric superego." Because moderation systems lack cultural nuance for low-resource languages, they systematically over-filter benign dialectal content, mistaking idioms for toxicity.

4. Key Results

The empirical analysis of tokenization reveals stark disparities:

Token Inflation Ratios:
- OpenAI (GPT-4.1): Arabic sentences require 1.62x more tokens than English equivalents. Darija (Arabic script) requires 1.59x, while Arabizi requires 1.86x.
- Mistral-7B (SentencePiece): The disparity is significantly higher. MSA Arabic requires 3.94x more tokens, and Darija requires 3.46x more than English.
Economic and Technical Impact: This "over-segmentation" (fertility) mechanically increases inference costs and consumes context window space, effectively pricing out or degrading performance for speakers of these languages.
Latent Space Effects: The paper demonstrates that minority languages suffer from anisotropy (uneven distribution in vector space) and dimensional collapse, where dialects are forced to normalize toward standard forms (MSA) or Western schemas, erasing cultural identity markers.
Safety Filter Failures: Moderation systems trained on Western norms frequently flag Darija and other dialects as toxic due to a lack of contextual understanding, creating a "double bind" where these languages are both underperforming and over-censored.

5. Significance and Implications

Theoretical Shift: The paper moves the debate from "fixing data" to "auditing architecture." It argues that bias is not just a reflection of human prejudice but an emergent property of the mathematical and architectural design of LLMs.
Critical AI Literacy: It highlights that users (especially students and researchers) are unknowingly participating in a system that shapes what can be thought and said, necessitating a "critical appropriation" of AI tools.
Proposed Solution: A Technical Clinic: The author proposes a "technical clinic" for models, analogous to a medical diagnosis rather than a therapy. This involves:
- Topological Audits: Mapping latent space curvature and tokenization fertility.
- Repair Strategies: Developing "tokenization parity" (fairer tokenizers), diversifying training data to include informal registers and hybrid scripts (e.g., Arabizi), and implementing multicultural alignment frameworks.
Ethical Urgency: As LLMs become core cognitive infrastructures, ignoring the algorithmic unconscious leads to "epistemic violence," where minority cultures are rendered unthinkable by the system itself.

In conclusion, Boisnard argues that the algorithmic unconscious is a structural reality that requires external intervention (audits, architectural changes) because the models themselves lack the capacity for self-reflection or correction.