The Thousand Brains Theory 2.0: An Extension for the Long-Range Connections of the Neocortical Heterarchy

Technical Summary: The Thousand Brains Theory 2.0: An Extension for the Long-Range Connections of the Neocortical Heterarchy

Problem Statement
The Thousand Brains Theory (TBT) 1.0 successfully proposed that individual cortical columns function as sensorimotor systems capable of learning structured models of objects by integrating sensory input with movement data in reference frames. However, the original theory did not fully address the functional roles of prominent long-range connections in the neocortex, specifically hierarchical feedforward and feedback connections, and cortico-thalamo-cortical (CTC) pathways. Furthermore, existing theoretical models of the neocortex predominantly emphasize a strict hierarchy where complex objects are only recognized after sequential processing through multiple layers. This hierarchical view struggles to explain experimental evidence showing that columns in "low-level" and "high-level" regions communicate directly, operate in parallel, and share non-hierarchical lateral connections. Consequently, the neocortex is better described as a heterarchy, yet a unified model explaining the computational purpose of these diverse long-range connections remains absent. Additionally, the original TBT lacked a mechanism for how information is converted from the egocentric perspective of sensors to the allocentric perspective of learned object models, and how the cortex learns compositional objects (objects made of other objects).

Methodology and Theoretical Framework
The authors extend the TBT by first reviewing the anatomy of long-range neocortical connections, categorizing them into four groups: hierarchical cortico-cortical (CC), non-hierarchical CC, cortico-thalamic, and Layer 5 motor connections. They argue that the mixture of hierarchical and non-hierarchical properties necessitates a heterarchical framework.

Building on the TBT 1.0 premise that every column learns structured models of objects using reference frames, the authors propose specific computational roles for the previously unaddressed connections:

1. Thalamic Transformations: They propose that thalamic relay cells perform a critical orientation transformation, converting sensory and movement data from an egocentric reference frame (relative to the sensor/body) to an allocentric reference frame (relative to the object model).
2. Compositional Learning: They propose that hierarchical feedforward and feedback connections are not for building complex features from simple primitives, but for learning compositional relationships (parent-child object structures).
3. Mechanism of Transformation: The authors suggest a "multiplexer" mechanism for thalamic relay cells, where cortical feedback (specifically from Layer 6b) selects which of multiple subcortical inputs is relayed to the cortex, effectively rotating the input to match the object's orientation.
4. Location-by-Location Learning: For compositional objects, the authors propose that parent-child relationships are learned on a location-by-location basis, where feedback connections associate specific locations on a parent object with corresponding locations on a child object.

Key Contributions
The paper makes three primary theoretical contributions:

• Thalamic Orientation Transform: The authors propose that the thalamus is not merely a passive relay but actively transforms the orientation of sensory features and movement data. Thalamic relay cells receive modulatory input from Layer 6b of the cortex, which encodes the inferred orientation of the sensor relative to the object. This allows the thalamus to rotate incoming egocentric data into the allocentric frame required by the cortical column's model. This mechanism explains how the brain maintains stable object recognition despite sensor rotation (e.g., head tilting) and rapid adaptation to prismatic glasses.
• Hierarchical Connections for Compositional Structure: The paper redefines the purpose of hierarchical connections. Instead of a feedforward accumulation of simple features, hierarchical connections enable columns to learn compositional objects. In this view, a lower region (e.g., R1) and a higher region (e.g., R2) both represent whole objects. The connections define the spatial relationship (location, orientation, and scale) between a "child" object (in R1) and a "parent" object (in R2). Feedback connections convey predictions about the expected object and its specific location on the parent.
• The Neocortex as a Heterarchy: The authors argue that the neocortex operates as a heterarchy. While hierarchical connections exist for learning compositional structure, regions also operate in parallel via direct driving inputs from the thalamus and non-hierarchical lateral connections. This allows columns at all levels to represent whole objects simultaneously, facilitating rapid consensus (voting) and multisensory binding without waiting for sequential hierarchical processing.

Results and Predictions
The paper does not present new experimental data but synthesizes existing empirical findings to support the proposed framework and generates specific, testable predictions:

• Support for Thalamic Transformation: The proposal aligns with findings that border ownership cells in V1 respond invariantly to object rotation (Sauvan and Peterhans, 1999) and that humans rapidly adapt to inverted vision (Stratton, 1897). The authors predict that if a familiar object is rotated, border ownership cells will maintain their firing patterns relative to the object, even if the retinal edge orientation changes.
• Support for Compositional Learning: The theory predicts that low-level sensory regions (like V1) can represent complex objects, not just simple features, consistent with recent decoding studies (Lanfranchi et al., 2025). It predicts that hierarchical feedback connections will show spatially clustered synapses in Layer 6 of the lower region, specifically linking columns with overlapping receptive fields.
• Thalamic Mechanism: The "multiplexer" model predicts that thalamic relay cells receive multiple driving inputs but activate only one at a time, controlled by cortical context. This contradicts the classical view of relay cells as simple one-to-one conduits.

Significance and Implications
The authors claim that this extended theory provides a unified explanation for the diverse long-range connections in the neocortex, resolving the discrepancy between hierarchical theories and heterarchical experimental data. By positing that the cortex learns compositional structures of whole objects rather than simple features, the theory offers a more biologically plausible account of robust object recognition and rapid learning.

The paper highlights significant implications for Artificial Intelligence (AI). Unlike current deep learning architectures that rely on backpropagation and sequential feature extraction, the TBT 2.0 suggests a path toward AI systems that:

1. Integrate sensor movement with features at every level to learn structured models of complete objects.
2. Use hierarchy to learn the compositional arrangement of objects in the world.
3. Learn via local updates within processing units rather than global gradient calculations.

The authors conclude that this framework offers a novel basis for understanding brain function and designing next-generation AI systems capable of continual learning, robustness to novel rotations, and principled model-based movement.