🧬 biology

Learning Options for Compositional Motor Control with Adapter Banks

This paper proposes a novel architecture using a shared recurrent core modulated by a bank of residual adapters to learn flexible motor primitives as emergent low-rank perturbations, enabling a frozen high-level policy to sequence these adapters for superior generalization to novel motor sequences compared to multitask baselines.

Original authors: Sreejan Kumar, Marcelo Mattar, Lea Duncker

Published 2026-09-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sreejan Kumar, Marcelo Mattar, Lea Duncker

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Skilled movement, from a pianist's fluid scales to a gymnast's complex vault, relies on a hidden ability: the brain does not learn every new motion from scratch. Instead, it builds a library of reusable building blocks, or primitives, that can be mixed, matched, and reshaped to create endless new actions. This capacity to recombine a finite set of skills into novel sequences is a hallmark of human dexterity. For decades, scientists have tried to teach artificial agents this same flexibility, hoping to create robots that can adapt to new tasks without needing to be retrained from the ground up. The challenge has been finding a way for a learning system to discover these building blocks on its own and then combine them without the old skills interfering with the new ones. Recent theories in neuroscience suggest that the brain might solve this by using a shared network of neurons that is slightly tweaked by different inputs, allowing the same core machinery to produce vastly different behaviors.

A new study by researchers at Columbia University and New York University brings this biological theory to life in a computer model, demonstrating how an artificial system can learn to compose complex movements from scratch. The team created a digital brain designed to control a simulated two-jointed arm with six muscles, much like a human limb. They asked this system to learn a variety of movements, including reaching straight, moving in circles, and tracing figure-eight patterns. Rather than simply memorizing these paths, the researchers wanted the system to discover the underlying parts of each movement and learn how to stitch them together to perform tasks it had never seen before. To do this, they built an architecture inspired by the connection between the brain's motor cortex and the thalamus, a deep brain structure that helps select and switch between different motor programs.

The researchers designed a system with a shared "core" network that handles the basic mechanics of movement, similar to the motor cortex. Attached to this core is a bank of specialized modules, or adapters, which act like small, adjustable knobs that can tweak the core's behavior. In the brain, these adapters would correspond to signals from the thalamus that slightly alter the activity of the motor cortex to produce a specific motion. The researchers did not program the system to know which adapter to use when; instead, they let the system learn to identify and select the right module by watching an expert teacher perform the tasks. The teacher was a separate, pre-trained network that knew exactly which rule to follow for each movement. The student system, however, had to figure out how to break the teacher's actions into segments and decide which adapter to activate at each moment, all without being told the rules explicitly.

As the student system trained on these compound movements, something remarkable happened. Even though the adapters were built to be fully flexible and complex, the system naturally learned to use them as simple, low-rank adjustments. In plain terms, the system discovered that it did not need to rewrite the entire brain for every new task; it only needed to make small, precise tweaks to the shared core. This allowed the system to factor the different movements into distinct, separate spaces within its internal state. While the teacher network, which relied on explicit rule inputs, kept all its task representations tangled together in one shared space, the student system organized them into highly separated zones. Each movement type, like a straight reach or a circular arc, occupied its own distinct corner of the system's internal landscape, making it much easier to switch between them without confusion.

The true test of this approach came when the researchers asked the system to perform a completely new task: a two-petal trajectory that combined parts of the movements it had learned in a way it had never seen before. This new path required stitching a straight extension to a curved retraction, and then a curved extension to a straight retraction, creating a discontinuity that neither the student nor the teacher had ever practiced. The researchers froze the internal networks of both systems and tried to find the best way to sequence the available building blocks to match this new target. The student system, with its separated and modular architecture, succeeded with high precision, tracking the new path with an error of about 0.005 meters. The teacher system, relying on its tangled, single-subspace approach, struggled significantly, plateauing with an error ten times larger.

The findings suggest that the key to flexible motor control is not just having a shared network, but having that network modulated by distinct, low-rank adjustments that keep different tasks in separate internal spaces. This geometric separation allows the system to mix and match its learned primitives to solve novel problems without the interference that usually plagues artificial learning. The study provides a concrete demonstration that an architecture inspired by the brain's thalamocortical loops can learn to discover its own reusable building blocks and recombine them to handle out-of-distribution challenges. By showing that a system can learn to factor its behavior into distinct subspaces, the research offers a promising path toward creating machines that can adapt to new physical tasks with the same ease and creativity as a human athlete.

Technical Summary: Learning Options for Compositional Motor Control with Adapter Banks

Problem Statement
Skilled motor control relies on the ability to generate flexible behavior from a finite set of reusable primitives that can be recombined and rescaled across new contexts. In artificial agents, a central challenge in hierarchical and modular control is discovering these reusable building blocks and recombining them to solve new problems without catastrophic interference. While the "options framework" in reinforcement learning addresses temporally extended actions, ensuring that learned options are truly compositional—allowing for novel sequences without retraining—remains difficult. Furthermore, neuroscience theory suggests that flexible motor control may rely on low-rank perturbations of a shared recurrent network (mimicking thalamocortical loops), but it remains unclear how such a system can be learned end-to-end from demonstrations to support genuine out-of-distribution (OOD) compositional transfer.

Methodology
The authors propose CABRA (Compositional Adapter Banks for Recurrent Architectures), a novel encoder-decoder architecture designed to learn motor skills end-to-end from compound demonstrations without explicit rule inputs.

Architecture: The system consists of a shared recurrent core (an LSTM) modulated by a bank of residual adapters. Each adapter is selected by a discrete latent code ( $z$ ) inferred from expert demonstrations.
Thalamocortical Inspiration: Inspired by Logiaco et al. [12], the architecture models motor primitives as low-rank perturbations ( $u_k v_k^\top$ ) of a shared cortical substrate ( $J_0$ ). Unlike previous theoretical works, the adapters here are not pre-defined; they are learned residual modulators ( $\Delta_k$ ) that perturb the core's hidden state.
Training Framework (Teacher-Student):
- Teacher: A rule-input-conditioned RNN trained on closed-loop biomechanical control (a two-link, six-muscle arm) to generate expert trajectories for various tasks (reaches, circles, figure-eights).
- Student Encoder: Reads the teacher's trajectory (excluding the rule input) and segments it into two parts (extension and retraction). It outputs a boundary distribution and a categorical distribution over $K=10$ latent codes, selecting one adapter per segment.
- Student Decoder: Executes the task in a closed-loop setting using only body-state feedback. The core network's recurrence is frozen during the decoder pass; gradients flow only to the adapters, muscle readout, and input projections.
Learning Objectives: The system is trained end-to-end using a hand-trajectory L1 loss against the teacher, KL regularization on code/boundary distributions, and auxiliary losses including a behavioral discriminator (DIAYN) and muscle excitation matching. Crucially, the student is trained only on compound trajectories (e.g., full reach-return), never on the half-task primitives, forcing the discovery of reusable sub-movements.

Key Results

Emergent Low-Rank Structure: Despite being initialized as full-rank residual modulators with no architectural rank constraints, the adapters learn to apply emergent low-rank perturbations. Quantitative analysis shows individual adapter perturbation subspaces converge to an effective rank of $\sim$ 5–6, while the union of all adapters spans a rank nearly matching the core's hidden state manifold. This reproduces the structural assumption of Logiaco et al. [12] as a learned property rather than an imposed constraint.
Geometric Factoring of Representations: The student network factors task representations into highly separated subspaces within the shared recurrent state space. In contrast, a baseline multitask teacher (modulated by rule inputs) collapses all task representations into a single shared manifold. The student's adapters introduce structured dynamical differences, creating distinct dynamical systems organized by behavioral kinematics (e.g., separating clockwise vs. counter-clockwise rotations).
Compositional Transfer: The primary demonstration of success is out-of-distribution compositional transfer. The authors constructed a novel "two-petal" trajectory by splicing primitive movements in combinations never seen during training.
- When optimizing a high-level policy over the frozen networks, the student successfully tracked the novel trajectory with an L1 error of $\approx$ 0.005.
- The baseline teacher plateaued at an error of $\approx$ 0.04 (an order of magnitude worse).
- The student's policy effectively sequenced existing adapters to handle kinematic discontinuities, whereas the teacher struggled to manage transitions within the novel splice-compositions.

Significance and Claims
The paper claims to bridge the gap between neuroscience theory and practical learning algorithms for motor control. By translating the principle of "low-rank perturbations of a shared network" into a learnable architecture, the authors demonstrate that:

End-to-End Learning of Primitives: A system can discover reusable, low-rank motor primitives from compound demonstrations without explicit supervision on the primitives themselves.
Compositional Generalization: The geometric factoring of task representations into separated subspaces enables the recombination of primitives into novel, out-of-distribution sequences with significantly lower error than task-input-conditioned baselines.
Biological Plausibility: The architecture provides a computational mapping to the cortex-thalamus-basal ganglia loop, where the shared core represents the cortex, adapters represent thalamic perturbations, and the latent code selection mimics basal ganglia action selection.

The authors position this work as a step toward resolving the dichotomy between high-level action selection and low-level kinematic specification in basal ganglia research, suggesting that selecting a specific low-rank perturbation simultaneously prescribes a specific kinematic pattern. They acknowledge limitations, noting that the current training relies on task-identity supervision for the encoder and that purely unsupervised versions of the architecture remain an open question for future work.

Technical Summary: Learning Options for Compositional Motor Control with Adapter Banks

More like this