Sustaining Control and Agency Under Threat: Computational Pathways to Persistence and Escape

This study introduces a novel persistence-escape paradigm and a Meta-Arbitration of Control and Agency Q-learning (MACA-Q) model to demonstrate that avoidance is a context-dependent, dynamically regulated response to inferred controllability rather than a stable trait, revealing distinct computational pathways for adaptive and maladaptive engagement in anxiety and depression.

The Gist

The Big Question: When to Keep Going, and When to Quit?

Imagine you are driving a car on a long, tricky road. Sometimes the road is smooth and sunny (Safe); other times, there are potholes, fog, and a storm (Threat).

At any moment, you have two choices:

1. Persist: Keep driving through the storm, trying to fix the car or navigate the bad road to get a big reward at the end. This takes effort and is risky.
2. Escape: Pull over, turn off the engine, and take a guaranteed, smaller snack from the glovebox. You give up the big reward, but you stop the stress immediately.

This paper asks a simple but profound question: How do we decide when to keep driving and when to pull over?

Most scientists used to think that when things get scary (threat), everyone just wants to pull over (escape) as fast as possible. But this study found that the answer is much more complicated. It depends on whether you feel like you are actually in control of the car.

---

The Experiment: A Video Game Mission

The researchers built a video game to test this. Imagine you are a delivery driver in a game.

• The Goal: Deliver a package to get points.
• The Choice:
  • Option A (Persist): Drive through a "code-cracking" tunnel. It's hard work. If you solve the puzzle, you get the package. If you fail, you might lose points. Crucially, your success depends on your skill.
  • Option B (Escape): Take a shortcut. You get a smaller package, but you don't have to work. However, you have no control here. The outcome is just luck (like rolling dice).

They played this game in different "worlds":

1. Safe World: If you fail, you just get zero points. No big deal.
2. Threat World: If you fail, you lose points you already had. This feels scary.
3. Broken Control World: Even if you try your best, the game is rigged so you can't win.

The Surprising Discovery

The classic theory said: "When the world gets scary (Threat), people should panic and quit (Escape) more often."

The study found the opposite.

When the threat was scary, but the driver still had control (they could actually solve the puzzle), they actually worked harder and stuck with the task more! They didn't want to quit. They felt that because they could control the outcome, the risk was worth taking.

However, if the driver felt they had lost control (the game was rigged), then they quit immediately.

The Analogy:
Think of it like a video game boss fight.

• If the boss is scary but you have a good weapon and a strategy, you get pumped up and fight harder.
• If the boss is scary and you realize your weapon is broken (no control), you immediately run away.

The "Agency" Concept: The Invisible Compass

The paper introduces a new idea called Agency.

• Self-Efficacy is thinking: "I am good at this game."
• Agency is thinking: "My actions actually change the outcome."

The study found that Agency is the most important factor. Even if you are good at the game, if you feel your actions don't matter, you quit. But if you feel your actions matter, you will brave the storm.

What About Anxiety and Depression?

The researchers looked at people with different levels of anxiety and depression to see how their "internal compasses" worked differently.

1. The Anxious Driver (High Anxiety)

• The Problem: They treat effort like a heavy, unmovable weight.
• The Behavior: Even if they know they can succeed, the fear of the effort makes them want to quit. They don't trust that their hard work will pay off. They are like a driver who sees a pothole and immediately pulls over, even if they have a spare tire and the road is fixable. They are too sensitive to the "cost" of trying.

2. The Depressed Driver (High Depression)

• The Problem: They have lost the belief that they can succeed, even when they are actually capable.
• The Behavior: They quit too easily because they think, "Why bother? I'll fail anyway." It's like a driver who sees a flat tire and assumes the car is totaled, even though they are perfectly capable of changing it. They have a "pessimistic filter" that blocks out their own competence.

The Solution: The "MACA-Q" Model

The scientists built a computer brain (a model called MACA-Q) to explain this.

Imagine a car with three layers of software:

1. The Engine (Learning): It learns how much points you get for driving.
2. The Dashboard (Feelings): It tracks how scary or safe the road feels.
3. The GPS (Meta-Control): This is the boss. It looks at the Engine and the Dashboard and asks: "Do I have control right now? Is it worth the gas?"

The model showed that healthy people use their GPS to switch strategies. If the road is bad but controllable, they drive. If the road is broken, they quit.

Anxious people have a GPS that is stuck on "High Cost." It screams "STOP!" even when the road is fine.
Depressed people have a GPS that is stuck on "No Control." It screams "STOP!" even when they are actually in control.

Why Does This Matter?

This study changes how we think about "giving up."

• Giving up isn't always bad. Sometimes, quitting is the smartest thing to do (like when you have no control).
• Resilience isn't just "gritting your teeth." Real resilience is knowing when to keep going and when to let go.
• Mental Health: Anxiety and depression aren't just about "being sad" or "being scared." They are glitches in the computer code that decides when to persist and when to escape.

The Takeaway:
To be adaptable, you need to know if you are the driver or just a passenger. If you are the driver, you can handle the storm. If you are just a passenger, it's okay to let someone else take the wheel or to stop the car. The goal isn't to never quit; it's to quit for the right reasons.

Technical Summary

1. Problem Statement

Adaptive behavior requires the ability to arbitrate between persistence (continuing effort despite costs) and escape (disengaging from a task). While avoidance is often studied as a response to threat or cost, existing paradigms fail to disentangle whether disengagement stems from:

• Threat sensitivity (anxiety).
• Reduced perceived controllability (learned helplessness).
• Effort aversion or feasibility expectations (depression).

Current models often conflate these mechanisms, treating avoidance as a global trait rather than a context-dependent policy. The authors argue that similar avoidance behaviors can arise from distinct computational pathways, necessitating a framework that separates self-efficacy (belief in one's capability) from agency (the belief that one's actions causally influence outcomes).

2. Methodology

A. Experimental Paradigm: Persistence-Escape Arbitration

The authors developed a novel, game-based task implemented on the Gearshift Fellowship platform with a large online sample (N = 457).

• Core Mechanism: On each trial, participants choose between Persistence (a code-cracking task requiring cognitive effort to control the outcome) and Escape (a probabilistic option yielding a fixed reward but relinquishing control).
• Contextual Manipulations: The task varies across blocks along three dimensions:
  1. Incentive Regime: Safe (gain vs. no gain) vs. Threat (gain vs. punishment/loss).
  2. Relative Value: Whether Escape or Persistence offers a higher objective monetary value.
  3. Controllability: Whether persistence outcomes are contingent on performance or randomly determined (Control Loss).
• Calibration: A separate "calibration mission" without escape options was used to estimate individual objective competence ($p_{Success}$), allowing the dissociation of ability from strategic disengagement.
• Clinical Measures: Anxiety and depressive symptoms were assessed using the DASS-21 (21-item Depression Anxiety Stress Scales).

B. Computational Modeling: MACA-Q

To explain the behavioral data, the authors introduced the Meta-Arbitration of Control and Agency Q-learning (MACA-Q) model. This is a hierarchical reinforcement learning model that extends standard Q-learning with three interacting layers:

1. RL Layer: Learns action values ($Q$) for persistence and escape via prediction errors.
2. Latent State Modules (The "Observer"):
   • Self-Efficacy ($SE$): Tracks expected success probability based on performance history.
   • Agency: A dynamically inferred gating signal that determines if outcomes are informative. It is computed based on the consistency between expected success and actual feedback. If agency is low (outcomes seem random), learning is suppressed.
   • Affective Valuation: Tracks reference-dependent emotional states (gain/loss relative to recent history) and reappraisal processes.
3. Meta-Control Layer (The "Governor"): Arbitrates between value-based learning, heuristic cues (stars, risk buses), and affective signals. It regulates how strongly context influences the decision policy.

C. Statistical & Validation Approach

• Behavioral Analysis: Linear mixed-effects models (LMM) tested interactions between symptom severity (anxiety/depression) and contextual factors (threat, value, controllability).
• Model Validation:
  • Predictive Checks: MACA-Q was compared against a canonical Q-learning baseline.
  • Parameter Recovery: Synthetic data was generated to ensure model parameters were identifiable.
  • Ablation Studies: Systematic removal of model components to verify their necessity.

3. Key Results

A. Behavioral Findings

• Threat Does Not Uniformly Increase Avoidance: Contrary to classical avoidance theories, participants persisted more under threat when outcomes were controllable. Escape rates only increased significantly when controllability was removed (Control Loss).
• Context-Dependent Arbitration: High-performing individuals were not uniformly more persistent; they were selective, escaping only when persistence was objectively disadvantageous or uncontrollable.
• Symptom Dimensions:
  • Anxiety: Associated with a disruption of decision arbitration. High-anxiety individuals treated effort as an unconditional cost, showing reduced sensitivity to incentive structures and value signals. They exhibited "threat overgeneralization," escaping even when persistence was advantageous.
  • Depression: Associated with distorted inputs to the decision process. High-depression individuals showed reduced differentiation between effort levels and held pessimistic beliefs about success ($p_{Success}$), leading to generalized escapism, particularly when success was unlikely.

B. Computational Findings

• Failure of Standard RL: Canonical Q-learning failed to capture key signatures, such as the reduced escape under threat (when controllable) and the gradual adaptation to context changes.
• MACA-Q Success: The MACA-Q model successfully reproduced all behavioral patterns, including the dissociation between self-efficacy and agency.
• Distinct Pathways to Avoidance: Simulations revealed that identical escape behaviors can arise from different computational mechanisms:
  • Threat-Sensitivity Route: Biases affective valuation toward persistence under threat without altering learning dynamics.
  • Reappraisal Route: Accelerates the downregulation of persistence after negative outcomes.
  • Meta-Control Route: Stabilizes agency accumulation, allowing flexible switching between persistence and escape based on context.
  • Learning Rate Route: High learning rates lead to volatile agency estimates and rapid disengagement.
• Accumulated Agency: Agency built during "safe" contexts (where failure is non-punitive) acts as a buffer, reducing maladaptive escape in subsequent threat contexts. This suggests resilience is the capacity to regulate persistence based on inferred controllability, not just a tendency to persist.

4. Key Contributions

1. Conceptual Distinction: Formally distinguishes Agency (causal influence of action) from Self-Efficacy (belief in capability), showing they are distinct latent variables that interact to guide behavior.
2. Novel Paradigm: Introduces a task that explicitly manipulates controllability and threat simultaneously, allowing the dissociation of effort costs from feasibility.
3. Computational Framework (MACA-Q): Provides a unified model integrating reinforcement learning, meta-control, affective valuation, and emotion regulation (reappraisal). It demonstrates that adaptive behavior requires a hierarchical arbitration of these signals.
4. Transdiagnostic Insight: Moves beyond global "avoidance" labels to show that anxiety and depression disrupt persistence-escape arbitration via distinct computational pathways (arbitration disruption vs. input distortion).

5. Significance and Implications

• Neuroscience: Suggests that prefrontal-basal ganglia circuits may implement a hierarchical control system where meta-control signals gate the influence of affective and value signals on action selection.
• Psychiatry: Offers a dimensional, mechanistic view of anxiety and depression. It implies that interventions (e.g., CBT, ACT) should target specific computational deficits (e.g., improving the inference of controllability for depression, or reducing threat overgeneralization for anxiety) rather than treating avoidance as a monolithic symptom.
• Artificial Intelligence: Proposes that adaptive agents must infer agency (whether their actions matter) to decide when to disengage. This prevents "blind persistence" in uncontrollable environments and enables efficient resource reallocation.
• Resilience: Reframes resilience not as the ability to endure all threats, but as the flexible regulation of engagement—persisting when control is possible and escaping when it is not.

In summary, the paper argues that avoidance is not a rigid bias but a dynamic policy regulated by meta-control processes that infer agency. Maladaptive behavior arises when these inference mechanisms are disrupted by specific symptom dimensions, leading to either rigid persistence or indiscriminate escape.