🧠 neuroscience

A neural circuit algorithm for action grammar

Motivated by monkey experiments, this study demonstrates that a recurrent neural network generates rule-based action sequences by planning initial states along orthogonal directions, executing shapes on specific manifolds, and preserving sequence parameters during transitions, thereby revealing a neural circuit implementation of a symbolic grammar algorithm.

Original authors: Liu, Y., Tian, L. Y., Tenenbaum, J. B., Freiwald, W. A., Wang, X.-J.

Published 2026-10-11

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Liu, Y., Tian, L. Y., Tenenbaum, J. B., Freiwald, W. A., Wang, X.-J.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

The human brain is a master of pattern, capable of taking a finite set of rules and generating an infinite variety of behaviors. This ability to follow abstract instructions—like the grammar that organizes words into sentences or the sequence that turns a series of movements into a coherent action—is a hallmark of intelligence. While modern artificial intelligence has made tremendous strides in recognizing patterns, it often struggles to apply learned rules flexibly to entirely new situations in the way humans and animals do. Scientists have long suspected that the frontal and parietal regions of the brain, areas associated with planning and organization, hold the key to this flexibility. However, the specific machinery inside these neural circuits that allows an animal to learn a rule and then apply it to a more complex problem has remained a mystery. Understanding how a biological network translates a simple instruction into a multi-step plan is essential for bridging the gap between the abstract logic of thought and the physical reality of neurons firing.

To uncover this mechanism, researchers turned to a task that monkeys have already mastered: drawing. In previous experiments, macaque monkeys were shown images containing multiple objects, such as lines and circles, and were asked to trace them in a specific order dictated by a rule, like "lines first, then circles." Remarkably, the monkeys were not explicitly taught the rule; they inferred it from experience. Once they learned the pattern, they could apply it to new images with more objects than they had ever seen before, successfully tracing five lines followed by three circles even if they had only practiced with two lines and one circle. This ability to generalize suggests the brain has an inbuilt bias for learning and using action grammars. To see how this works under the hood, the research team trained a computer model, specifically a recurrent neural network, to perform this same drawing task. By reverse-engineering the model's internal activity, they discovered a precise, three-part algorithm that the network uses to plan and execute these sequences, offering a concrete blueprint for how the brain might solve similar problems.

The researchers found that the network does not simply react to the image as it draws; instead, it formulates a complete plan before the first stroke is made. When the task begins, the network's internal state shifts to a specific starting position that encodes the entire future sequence. Imagine the network's activity as a point in a vast, multi-dimensional space. Before drawing starts, this point moves to a location that simultaneously holds two pieces of information: how many lines need to be drawn and how many circles need to be drawn. Crucially, these two numbers are stored along separate, independent directions within this space, much like how a map uses north-south and east-west axes to locate a point. This initial positioning acts as a high-level blueprint, ensuring the network knows exactly how many repetitions of each shape are required before it even begins to move.

Once the drawing begins, the network executes the plan by moving along a specific path dedicated to the current shape. If the rule is to draw lines first, the network's activity travels along a "line" path, repeating the action for as long as the distance to the end of that path allows. The network does not count out loud; rather, the distance it has to travel along this path corresponds directly to the number of repetitions it must perform. As it moves, it gradually consumes the distance until it reaches the end of the line path. At this precise moment, the network transitions to the next phase. It jumps to a different path dedicated to circles. The brilliance of this system lies in how it handles the transition: the jump is so orderly and parallel that it preserves the information about how many circles are left to draw. The network arrives at the start of the circle path at a distance from the end that perfectly matches the number of circles it needs to produce, allowing it to continue the sequence without losing track of the count.

This mechanism relies on three distinct types of neural activity working in concert. Some neurons are dedicated to identifying which shape is currently being drawn, effectively switching the network's mode from "line" to "circle." Others track the progress of the current stroke, signaling how many repetitions remain. Finally, a specific group of neurons becomes highly active right at the boundary between shapes, acting as a trigger that drives the network from one path to the next. The researchers observed that as the network nears the end of a sequence of lines, the inhibition that usually suppresses the "circle" neurons weakens, allowing them to fire up and take over. This seamless handoff ensures that the count for the upcoming shape is never lost during the switch. The study suggests that the brain uses a similar strategy of orthogonal planning and parallel transitions to organize complex behaviors, allowing it to generate endless variations of action from a simple set of rules.

The implications of this discovery extend beyond the drawing task. The mechanism the researchers uncovered provides a tangible explanation for how the brain can hold a mental program in place while executing its steps. It shows that the brain does not need to constantly re-calculate the entire sequence from scratch; instead, it sets an initial state that contains the full plan and then lets the dynamics of the neural circuit run that plan out. This approach allows for remarkable flexibility: if the problem changes and requires more repetitions, the network simply starts further out along the same paths, using the same underlying machinery. While the current study used a computer simulation to demonstrate this, the specific patterns of activity predicted by the model—such as the orthogonal encoding of numbers and the parallel transitions between shapes—offer clear, testable predictions for future experiments in living brains. By revealing how a neural circuit can implement a symbolic algorithm, this work brings us closer to understanding the biological basis of human intelligence and the ability to learn, adapt, and create.

Technical Summary: A Neural Circuit Algorithm for Action Grammar

Problem Statement
The paper addresses a fundamental question in cognitive science and neuroscience: how neural circuits implement abstract grammatical rules to generate sequences of discrete elements, such as actions or words. While modern artificial intelligence has advanced significantly, humans and animals remain superior at flexibly applying learned rules to novel problems. Specifically, the authors investigate how the brain organizes cognitive steps into a mental program using abstract grammatical rules, focusing on the generation of action sequences in the form $A^mB^n$ (producing shape $A$ , $m$ times, followed by shape $B$ , $n$ times). This builds upon recent behavioral findings where macaque monkeys learned to draw multi-object images according to such grammatical rules without explicit training on the grammar itself, demonstrating an inductive bias toward learning and using action grammars.

Methodology
To explore the neural circuit mechanisms underlying this ability, the authors trained a recurrent neural network (RNN) with $N=1024$ units to perform a simplified version of the monkey drawing task.

Task Design: The RNN received binary inputs representing the locations of lines and circles on a $4\times4$ grid and a rule input specifying the action grammar (either "lines $\to$ circles" or "circles $\to$ lines"). The network was tasked with outputting a sequence of strokes (shape and location) consistent with the rule, producing strokes from top-to-bottom and left-to-right.
Training and Generalization: The RNN was trained using supervised learning and backpropagation through time on problems containing 2–5 objects. Crucially, the model was then tested on generalization problems containing 6–8 objects, a regime not seen during training, to assess its ability to extrapolate the learned grammar.
Analysis: The authors employed dimensionality reduction techniques to visualize the RNN's high-dimensional state space. They identified three specific axes via linear regression: a "shape axis" (discriminating between lines and circles) and two orthogonal "repeat axes" ("repeat line" and "repeat circle") that encoded the number of remaining repetitions for each shape. They further analyzed neural trajectories, input weights, and single-unit responses to reverse-engineer the algorithm.

Key Results
The trained RNN successfully generalized to novel, more complex problems (6–8 objects), mirroring the behavioral performance of monkeys. Reverse-engineering the network dynamics revealed a specific three-part algorithm:

Orthogonal Initial State Encoding: Before drawing begins, the RNN state moves to a specific initial location determined by the external input. This initial state encodes the number of lines ( $m$ ) and circles ( $n$ ) along two orthogonal directions ("repeat line" and "repeat circle" axes). The distance of the state from the edge of the respective manifold corresponds to the number of repetitions required. Input weight analysis showed that connections from shape-location inputs are aligned with their corresponding repeat axes, while grammar inputs align with the shape axis, effectively "pushing" the state to the correct starting point.
Manifold Traversal for Repetition: During the execution of a shape, the RNN state progresses along the corresponding shape-specific manifold (e.g., the "repeat circle" manifold) toward its edge. This movement represents the decrementing count of remaining repetitions.
Parallel Transitions Preserving Information: When the network transitions from one shape to the next (e.g., from the last circle to the first line), the neural trajectories follow parallel paths across trials with different repetition counts. This parallel dynamics ensures that the information regarding the number of repetitions for the upcoming shape (encoded along the orthogonal axis) is preserved during the transition. Consequently, the state lands on the new manifold at a distance from the edge that correctly reflects the remaining count for the second shape.

At the single-unit level, the network contains distinct populations encoding: (1) the current shape, (2) the ordinal rank of the stroke within the repetition, and (3) the sequence boundary (active just before transition). The transition is driven by a gradual weakening of inhibitory input from units selective for the first shape to those selective for the second.

Significance and Claims
The authors claim that this work reveals a neural circuit implementation of a symbolic algorithm for generating rule-based action sequences. The proposed mechanism bridges the gap between symbolic algorithms (specifically the $A^mB^n$ structure) and their realization in neural networks.

Key claims regarding the mechanism's significance include:

Generalization via Initial State: The network generalizes to longer, novel sequences not by re-learning the rules, but by starting further along the same orthogonal directions based on the new input counts.
Distance-Based vs. Speed-Based: The authors argue that their observed "distance-based" mechanism (encoding count as distance to the manifold edge) is more efficient for generalization than a hypothetical "speed-based" mechanism, as it allows dynamics to be reused across problems without rescaling.
Neural Basis of Mental Programs: The results provide a concrete mechanism for how neural circuits organize processing steps to implement a "mental program," offering an account for how pre-movement activity patterns (encoding abstract structural properties) combine with subsequent dynamics to execute action grammars.
Alignment with Physiology: The identified single-unit responses (encoding shape, ordinal rank, and sequence boundaries) align with empirical observations in the frontal and premotor cortices of monkeys performing similar sequencing tasks, suggesting a plausible biological substrate for these computational principles.

The paper concludes that while this mechanism explains the generation of $A^mB^n$ sequences, future work is needed to understand how such primitives are assembled into more complex, hierarchical grammars found in natural language and advanced cognition.

Technical Summary: A Neural Circuit Algorithm for Action Grammar

More like this