💻 computer science

Functional Misalignment in Human-AI Interactions on Digital Platforms

This paper argues that the adverse societal outcomes of algorithmic systems, such as polarization and mental health issues, stem from a structural "functional misalignment" where optimizing for predictable behavioral signals fails to align with genuine human goals, a problem driven by biases toward reactive signals, feedback loops, and emergent collective dynamics.

Original authors: Kristina Lerman

Published 2026-04-15

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Kristina Lerman

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Perfect" Chef Who Serves Junk Food

Imagine you hire a super-smart chef (the Algorithm) to cook meals for you. Your goal is to be healthy, happy, and satisfied in the long run.

The chef is incredibly talented at one specific thing: predicting what you will eat next.

If you grab a donut, the chef notes, "Ah, they like donuts!" and makes more donuts.
If you stare at a spicy taco for a second, the chef thinks, "They love spice!" and serves you a volcano of hot sauce.
If you click on a video of a cat falling off a table, the chef assumes, "They want chaos!" and shows you a thousand videos of falling cats.

The chef is perfectly accurate at predicting what you will click on or eat. But here is the problem: You don't actually want to eat only donuts, hot sauce, and falling cats. You want a balanced diet that makes you feel good tomorrow, not just right now.

This paper argues that social media algorithms are exactly like this chef. They are so good at predicting your impulses (what you click) that they accidentally ignore your real needs (what makes you happy or healthy). The paper calls this "Functional Misalignment."

Why Does This Happen? (The Three Mechanisms)

The author explains that this mismatch happens because of three specific reasons:

1. The "Fast Brain" vs. The "Slow Brain"

The Analogy: Imagine your brain has two modes.
- System 1 (The Fast Brain): This is your reflex. It's the part that screams "LOOK!" at a bright red sign, gets angry at a rude comment, or feels jealous seeing a friend's vacation photo. It's fast, emotional, and automatic.
- System 2 (The Slow Brain): This is your thoughtful self. It's the part that says, "I should read this long article," or "I shouldn't scroll for three hours," or "I value kindness." It takes effort and time.
The Problem: Algorithms are like a chef who only listens to your Fast Brain. It's easy to predict when you'll grab a donut (Fast), but hard to predict when you'll choose a salad (Slow). So, the algorithm feeds you endless donuts because that's what your reflexes tell it you want, even if your thoughtful self is screaming for a salad.

2. The "Echo Chamber" Feedback Loop

The Analogy: Imagine a microphone that is too close to a speaker.
- You say something small (a click).
- The speaker (the algorithm) amplifies it and plays it back louder.
- You hear it louder, so you react even more strongly.
- The speaker amplifies it again.
The Problem: This creates a feedback loop. If you get angry at one political post, the algorithm shows you ten more angry posts. You get angrier, click more, and the algorithm learns, "Anger works! Let's give them more!" Soon, you are living in a world of pure anger, even if you started out just mildly annoyed. The system gets stuck in a loop of amplifying the most extreme emotions because they get the biggest reaction.

3. The "Crowd Effect" (Emergent Chaos)

The Analogy: Imagine a stadium where everyone is trying to get a better view.
- If one person stands up to see, everyone else stands up too.
- Soon, everyone is standing, and no one can see better than before, but everyone is tired and uncomfortable.
The Problem: When millions of people interact with these algorithms, small individual reactions (like clicking on a scary headline) add up to create massive societal problems. The system isn't trying to make the world polarized or anxious; it's just trying to get clicks. But because everyone is reacting to the same "fast brain" triggers, the whole society ends up more divided, anxious, and unhappy.

The Real-World Consequences

The paper shows how this "misalignment" causes real harm in three main areas:

Political Polarization: The algorithm learns that anger gets clicks. So, it shows you content that makes you hate the "other side." You start seeing your political opponents as monsters, not just people with different ideas. The algorithm didn't want to divide you; it just wanted to keep you clicking, and anger is the most reliable way to do that.
Mental Health (Especially for Teens): The algorithm learns that envy gets clicks. It shows teens photos of "perfect" bodies and "perfect" lives. Teens' "Fast Brains" react with jealousy and insecurity ("Why isn't my life like that?"). The algorithm keeps showing more of these images because the teens keep looking. This creates a cycle of depression and body image issues.
Bad Group Decisions: Imagine a group of people trying to pick the best song. If the algorithm shows them the songs that are already popular (because they got clicks), everyone just clicks those songs again. The "best" song might never get heard because it didn't get the first few clicks. The group ends up with a bad playlist, not because they are stupid, but because the system amplified the wrong signal.

Why "Fixing" It Is Hard

The paper argues that we can't just fix this by:

Getting more data: Knowing more about what you click doesn't help if you are clicking the wrong things.
Making better predictions: If the algorithm predicts perfectly that you will click on a hate-filled video, it has "succeeded" at its job, even though the result is bad for society.
Being transparent: Even if the app tells you, "We show you angry videos because they get clicks," you are still stuck in the loop. Your "Fast Brain" will still react to the anger.

The Solution: Change the Rules, Not Just the Chef

The paper suggests we need to stop treating algorithms like simple tools that just "predict." We need to treat them like complex systems that shape human behavior.

Instead of asking, "How do we get more clicks?" we need to ask:

"How do we design a system that values long-term well-being over short-term clicks?"
"How do we break the feedback loops that amplify anger and envy?"
"How do we measure if people are actually happy, not just engaged?"

In short: The problem isn't that the algorithms are "evil" or "broken." The problem is that they are working exactly as designed: they are masters at predicting our impulses, but they have forgotten to serve our better selves. To fix it, we have to change the rules of the game, not just the players.

1. Problem Statement

The paper addresses a critical paradox in digital platforms: algorithmic systems (particularly social media recommenders) have achieved unprecedented success in predicting and shaping human behavior (e.g., clicks, views, engagement), yet their widespread adoption correlates with severe negative societal outcomes. These include rising mental health crises among adolescents, increasing political polarization, erosion of democratic trust, and collective dysfunction.

The core problem is Functional Misalignment. The author argues that these adverse outcomes are not accidental failures but emergent properties of a structural mismatch between:

What algorithms optimize: The accurate prediction of observable behavioral signals (System 1 responses: fast, reactive, emotional, heuristic-driven).
What human goals actually are: Reflective judgment, long-term well-being, values, and needs (System 2 processes).

Current systems treat behavioral data (what users do) as a perfect proxy for user welfare (what users want), creating an "epistemic gap" where maximizing engagement systematically diverges from maximizing human welfare.

2. Methodology and Theoretical Framework

The paper does not present a single empirical experiment but rather synthesizes a unifying theoretical framework by integrating insights from three distinct disciplines:

Cognitive Science: Utilizing Dual-Process Theory (System 1 vs. System 2) to explain human judgment. It posits that algorithms disproportionately learn from System 1 signals (fast, emotional, biased) because they are more frequent and predictable than reflective System 2 judgments.
Machine Learning: Analyzing how recommender systems optimize for predictive accuracy over behavioral proxies (clicks, dwell time), inadvertently learning to trigger heuristic-driven responses rather than serving latent user values.
Complex Systems Theory: Modeling human-AI interaction as a dynamical system driven by feedback loops. It applies concepts like positive feedback, path dependence, and the "rich-get-richer" effect to explain how local optimizations lead to global instabilities.

The Framework: The paper proposes "Functional Misalignment" as a framework driven by three interacting mechanisms (illustrated in Figure 1 of the paper):

Observability Mismatch: Algorithms prioritize signals that are easy to observe and predict (System 1), ignoring harder-to-measure reflective preferences.
Feedback-Loop Dynamics: A closed loop exists where Algorithm $\to$ User Behavior $\to$ New Data $\to$ Algorithm. This creates self-reinforcing cycles that amplify small initial biases into large disparities.
Emergent Collective Effects: At scale, these loops produce systemic phenomena (polarization, norm distortion) that are unintended consequences of local optimization.

3. Key Contributions

The paper makes three primary contributions to the field of Human-AI Interaction (HAI):

A Unifying Framework: It moves beyond treating issues like polarization, radicalization, and mental health harms as isolated problems. Instead, it identifies them as shared structural consequences of the same underlying mechanism: the misalignment between optimization objectives and human welfare.
Mechanistic Explanation: It explicitly details how misalignment arises, bridging the gap between cognitive biases (human side) and algorithmic learning (AI side). It explains why accurate individual-level predictions can coexist with adverse societal outcomes.
Research and Policy Agenda: It outlines a new direction for HAI research, shifting focus from isolated interventions to system-level control and dynamic modeling.

4. Results and Pathological Outcomes

The paper analyzes three domains where functional misalignment manifests most severely:

Political Polarization:
- Mechanism: Algorithms amplify intergroup hostility because anger and moral outrage (System 1 responses) generate high engagement.
- Outcome: This creates "affective polarization," where political opponents are viewed as existential threats. The system sustains this by surfacing content that triggers tribal identification, making polarization an emergent property of the feedback loop rather than just ideological sorting.
Mental Health and Well-being:
- Mechanism: Algorithms exploit cognitive heuristics like "prestige bias" (mimicking high-status individuals).
- Outcome: Users are exposed to idealized images and lifestyles, triggering upward social comparison, envy, and body dissatisfaction. This feedback loop is linked to rising rates of anxiety, depression, and eating disorders, particularly among adolescents.
Crowdsourcing and Collective Decision-Making:
- Mechanism: Even without personalization, ranking based on popularity creates feedback loops (e.g., the MusicLab experiment).
- Outcome: Early random variations are amplified, leading to inequality (few items dominate) and unpredictability. Quality becomes a poor predictor of success, destabilizing collective intelligence.

5. Significance and Implications

The paper challenges dominant assumptions in industry and policy regarding algorithmic harms:

Against "More Data": The problem is not a lack of data about user behavior; the problem is that behavioral data is a biased proxy for user needs. More tracking of System 1 behavior will not solve the misalignment.
Against "Better Prediction": Improving predictive accuracy within a misaligned objective function (maximizing engagement) does not reduce harm; it accelerates it. The issue is the objective function itself, not the model's precision.
Against "Transparency": Transparency alone is insufficient. A fully transparent algorithm that optimizes for engagement remains misaligned with user welfare.

Proposed Solutions and Research Agenda:

Measurement: Researchers must distinguish between behavior, preference, and value. New methods are needed to elicit reflective (System 2) preferences (e.g., via surveys or deliberative prompts) rather than relying solely on behavioral traces.
Experimental Design:
- System 1 vs. System 2 Elicitation: Developing methods to separate impulsive signals from deliberative ones.
- Prosocial Feed Design: Testing ranking objectives that prioritize diversity and well-being over raw engagement.
- Control of Feedback Dynamics: Treating HAI as a control problem to steer system trajectories toward desirable equilibria, potentially using network-based interventions at key nodes.
Policy: Regulation must move beyond detecting bad actors to redesigning structural incentives. This involves placing limits on optimization itself (e.g., constraining feedback architectures) to protect values that cannot be achieved through efficient optimization alone.

In conclusion, the paper argues that functional misalignment is a structural feature of current digital ecosystems, requiring a fundamental shift from optimizing for engagement to designing systems that respect the complexity of human cognition and collective welfare.