🧬 biology

Low-dimensional factorized neural computations underlie risk-adaptive choices

This study demonstrates that risk-adaptive decision-making relies on low-dimensional neural computations that segregate safe and risky states, a mechanism identified through deep reinforcement learning agents and validated by similar dynamical patterns in human neuronal recordings.

Original authors: Elliot Smith, T Price, Andrew Liu, Rhiannon Cowan, Niloufar Shahdoust, Tyler Davis, Bornali Kundu, John Rolston, Shervin Rahimpour, Ben Shofty, Alla Borisyuk

Published 2026-08-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Elliot Smith, T Price, Andrew Liu, Rhiannon Cowan, Niloufar Shahdoust, Tyler Davis, Bornali Kundu, John Rolston, Shervin Rahimpour, Ben Shofty, Alla Borisyuk

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine your brain as a super-smart navigation system trying to get you through a foggy forest. Sometimes the path is clear, but often, you have to guess: "If I take this shortcut, will I find a treasure chest, or will I fall into a pit?" This is the daily reality of decision-making. Scientists call the study of how we learn from these guesses Reinforcement Learning. It's the same logic a dog uses when learning tricks (treats for good behavior) or a video game character uses to level up. But here's the tricky part: real life isn't a video game with perfect instructions. We have to weigh the thrill of a big reward against the scary chance of a big loss. When this system glitches, it can lead to trouble, like gambling addiction or impulsive choices. To understand how our brains handle this high-stakes guessing game, researchers are now teaming up with artificial intelligence, using computer programs to simulate how a brain might think, and then checking if real human brains actually work that way.

This paper is a fascinating detective story where the investigators used a swarm of computer agents to solve a risky game called the "Balloon Analog Risk Task" (BART). In this game, you pump up a balloon; the bigger it gets, the more points you win, but if you pump it too much, it pops and you get nothing. The researchers didn't just watch the computers play; they watched how the computers' "brains" (their internal neural networks) changed while they learned. They discovered that the computers naturally split into two distinct personality types. One group, the "Explorers," was super cautious. They pumped the balloons very slowly and carefully, trying to figure out every single detail of the game, but they ended up with fewer points. The other group, the "Bimodal" strategists, were risk-adaptive masters. They learned to pump small balloons just a little bit (to be safe) but pumped big balloons way more, knowing exactly when to stop. These "Bimodal" agents were much better at the game.

But the real magic happened when the researchers looked at the shape of the thoughts inside these computer brains. They found that the "Bimodal" agents organized their thoughts in a neat, low-dimensional way. Imagine their thoughts as a smooth, curved slide that clearly separated "safe" from "risky" situations. The "Explorer" agents, on the other hand, had messy, tangled thoughts where safe and risky situations got all mixed up. The researchers then took this computer-generated map and compared it to the actual brain activity of 44 human patients with epilepsy who were playing the same balloon game. The results were strikingly similar. The humans who played like the "Bimodal" agents had brain activity that formed those same neat, organized slides, clearly separating the different levels of risk. The humans who played like the "Explorers" had the messy, tangled brain patterns.

The paper suggests that the secret to making smart, risky choices isn't just about having a "good" brain, but about having a brain that organizes information in a specific, efficient geometric shape. The "Bimodal" humans and computers both used a strategy where they treated medium and high-reward balloons as one big "go for it" category, while keeping the dangerous ones totally separate. This allowed them to make quick, adaptive decisions. In contrast, the "Explorers" collapsed the differences between the safe and medium risks, making it harder to decide when to take a chance. The study didn't prove that this is the only way the brain works, but it strongly suggests that our brains, like our best artificial intelligence, rely on these clean, low-dimensional maps to navigate uncertainty. It's a beautiful example of how the messy, complex world of human choices might actually be built on simple, elegant mathematical structures, whether in silicon chips or human neurons.

Technical Summary: Low-dimensional factorized neural computations underlie risk-adaptive choices

Problem Statement
Real-world decision-making frequently occurs under conditions of uncertainty, requiring individuals to weigh potential rewards against the probability of adverse outcomes. Failures in this process are linked to maladaptive behaviors and psychiatric disorders such as gambling addiction and substance use. While reinforcement learning (RL) provides a framework for modeling these decisions, the specific neural computations and population dynamics that facilitate inference about uncertain outcome landscapes remain unclear. The authors investigate whether deep RL agents and human brains converge on similar internal representations when solving risky decision-making tasks, specifically examining if distinct behavioral strategies are supported by distinct low-dimensional neural geometries.

Methodology
The study employed a "neuro-AI" approach, bridging artificial and biological intelligence through the Balloon Analog Risk Task (BART).

Deep RL Agents: The authors trained 400 randomly initialized deep RL agents using an Actor-Critic architecture with Proximal Policy Optimization (PPO). The agents were trained on "meta-BART," a variation where the mean balloon size ( $\mu$ ) varied across epochs. Agents were evaluated on a suite of 17 fixed $\mu$ conditions.
Agent Classification: Recurrent layer node activities were analyzed using k-means clustering (initially $k=12$ , reduced to 6 after symmetry adjustments). Agents were classified into three types based on node proportions. Two distinct behavioral phenotypes emerged: "Type 1" (later termed Bimodal) and "Type 3" (later termed Explorer).
Neural Dynamics Analysis: The authors analyzed the trajectories of agent recurrent layer activations in 3D Principal Component Analysis (PCA) space. They estimated a "balloon size axis" using Partial Least Squares (PLS) regression to project neural trajectories and assess their geometric structure (e.g., convex hulls, bimodality).
Human Data Collection: 44 patients with drug-resistant epilepsy undergoing intracranial monitoring (stereo-electroencephalography with microwires) performed a computerized BART. Single-unit activity was recorded from the medial frontal cortex (ACC, vmPFC) and mesial temporal lobe (MTL).
Human Classification: Human participants were clustered into "Bimodal" and "Explorer" groups based on behavioral metrics (total score, pop rate, and mean inflation times across red, orange, and yellow balloon distributions).
Cross-Domain Comparison: Neuronal ensembles from human participants were grouped into "pseudoensembles" corresponding to the Bimodal and Explorer behavioral strategies. These were analyzed using the same PCA and PLS methods applied to the agents to compare neural trajectory geometries, representational similarity, and variance explained.

Key Contributions and Results

Emergence of Distinct Strategies in Agents:
- Bimodal Agents (Type 1): These agents adopted an efficient, risk-adaptive strategy. They inflated smaller balloons less and larger balloons more, resulting in higher total scores without a significantly higher pop rate. Their neural trajectories formed low-dimensional representations that segregated safe and risky states. Specifically, their trajectories clustered bimodally near the edges of the balloon size axis, and the size axis aligned closely with the first principal component (PC1).
- Explorer Agents (Type 3): These agents exhibited a cautious, less successful strategy, slowly exploring the reward contingency space. Their neural trajectories were skewed, unimodal, and entangled, with the size axis organized orthogonally to PC1.
Prediction of Human Behavior:
- Human participants naturally segregated into Bimodal and Explorer groups with proportions similar to the agent distribution.
- Bimodal Humans: Like their artificial counterparts, they achieved significantly higher scores and longer inflation times for high-reward (yellow) balloons compared to low-reward (red) ones.
- Explorer Humans: They showed a more gradual increase in inflation times across balloon types and lower overall performance.
Convergence of Neural Geometries:
- Low-Dimensional Factorization: Bimodal human neuronal ensembles exhibited neural trajectories remarkably similar to Bimodal agents. These trajectories arced around an ordered balloon size axis that clearly segregated the expected value of different balloon colors. The 2-D convex hulls of these trajectories increased for balloons with larger mean inflations, indicating a robust predictive model.
- Entanglement in Explorers: In contrast, Explorer human ensembles lacked this structured, low-dimensional geometry. Their trajectories did not show the same coactivity patterns or clear segregation along the size axis.
- Representational Similarity Analysis (RSA): RSA revealed that Bimodal neural activity patterns significantly correlated with predicted balloon sizes (encoding physical size relationships), whereas Explorer patterns showed negative or absent correlations.
- Decision Boundaries: Pairwise centroid distances indicated that Bimodal humans compressed their state space around behaviorally relevant boundaries, treating medium and high-value balloons as a unified "inflate-worthy" category (Red distinct from Orange/Yellow). Explorers collapsed Red and Orange representations while separating Yellow.
Anatomical Contributions:
- In Bimodal participants, the ventromedial prefrontal cortex (vmPFC) accounted for the largest variance in PC1 (72.9%), suggesting prospective consideration of value/risk categories.
- In Explorer participants, the anterior cingulate cortex (ACC) accounted for a larger proportion of variance in PC1 (56.0%), potentially indicating a focus on monitoring reward prediction errors rather than building a task model.
- Control analyses confirmed that these geometric differences were not artifacts of sample size disparities or uneven anatomical sampling.

Significance
The paper claims that specific dynamical computations, characterized by low-dimensional, factorized neural manifolds, underlie successful inference about uncertain outcomes. The study demonstrates that deep RL agents can spontaneously generate distinct behavioral phenotypes and associated neural geometries that mirror those found in human neuronal ensembles.

The primary significance lies in the convergence of artificial and biological systems: the "Bimodal" strategy, which supports efficient risk management, is underpinned by untangled, low-dimensional trajectories that allow for organized readout of neural activity. This suggests that the human brain compresses complex probabilistic state spaces into low-dimensional dynamics on manifolds to simplify decision-making. The findings support the "Neuro-AI" hypothesis that artificial networks can serve as valid models for unpacking the dynamical computations underlying high-level cognitive processes, specifically distinguishing between model-based inference (Bimodal) and model-free or error-monitoring strategies (Explorer). The authors note limitations regarding the specific brain regions sampled (due to clinical constraints in epilepsy patients) and the abstraction of the RL agents, but conclude that aligning latent spaces across biological and artificial agents reveals shared algorithmic principles of decision-making.

More like this