🤖 machine learning

Learning Patterns and Abstractions from Perceptual Sequences

This thesis proposes that chunking and abstraction serve as fundamental computational principles enabling humans and models to efficiently learn, generalize, and predict structured patterns by breaking down perceptual sequences into reusable hierarchical primitives.

Original authors: Shuchen Wu

Published 2026-06-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shuchen Wu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: From Chaos to Order

Imagine your brain is a person walking through a chaotic, noisy city. There are flashing lights, honking cars, smells of food, and crowds of people. It's a mess of raw data. Yet, you don't see a blur; you see "traffic lights," "a hot dog stand," and "a red car." You instantly organize the chaos into useful, named objects.

This thesis asks: How does the brain do this? How do we turn a endless stream of noise into a structured world we can understand, remember, and predict?

The author argues that the secret ingredient is Chunking.

Part 1: The Art of "Chunking" (Grouping)

The Analogy: The Magic of the Zipper
Imagine trying to zip up a jacket with 100 tiny, individual teeth. It would take forever. But if you group those teeth into 10 larger "chunks" of teeth, you can zip it up in seconds.

The paper suggests our brains do the same thing with sequences of events (like a song, a sentence, or a series of button presses).

The Experiment: Participants played a game where they had to press keys in response to lights. Sometimes the lights followed a hidden pattern (like A-B-C-A-B-C).
The Finding: Humans are incredibly good at spotting these patterns. When they realize "Oh, A-B-C is a group," they stop pressing keys one by one and start pressing the whole group at once.
The Trade-off: The paper found that humans are smart about how they group things. If you tell them to go fast, they group things into bigger chunks (even if they make a few mistakes). If you tell them to be perfect, they group things into smaller, safer chunks. It's a rational balance between speed and accuracy.

The Takeaway: We don't just memorize lists; we bundle items into "action primitives" (like a single move in a dance) to save mental energy.

Part 2: Building Castles from Bricks (Composition)

The Analogy: LEGO Blocks
Once you have learned a few chunks (like the word "Black" and the word "Forest"), you can snap them together to make a bigger chunk ("Black Forest"). Then you can snap "Black Forest" and "Cake" together to make "Black Forest Cake."

The thesis proposes a computer model called HCM (Hierarchical Chunking Model) that does exactly this.

How it works: It starts by finding small, repeating patterns in a stream of data. Once it finds them, it treats them like single LEGO bricks. Then, it looks for patterns where those bricks appear together and snaps them into bigger bricks.
The Result: This allows the model to understand complex structures (like a whole book or a long movie) by breaking them down into nested layers of smaller, familiar parts.
Why it matters: This explains how we can learn infinite new things using a finite number of building blocks. It's how we can understand a sentence we've never heard before because we recognize the familiar "bricks" (words and phrases) inside it.

Part 3: The Abstract Leap (Seeing the Pattern, Not the Paint)

The Analogy: The Musical Score
Imagine hearing a song played on a piano, then the same song played on a guitar, then a flute. The notes (the concrete data) are different, but the melody (the abstract pattern) is the same.

The paper explores how humans can ignore the specific details (the instrument) and learn the underlying rule (the melody).

The Experiment: People were asked to memorize long sequences of colors. Some sequences had a hidden rule, like "Red, Blue, Red, Blue" (a pattern) or "Red, [Any Color], Red, [Any Color]" (a variable rule).
The Finding: Humans didn't just memorize the specific colors; they learned the structure. They could take that structure and apply it to a completely new set of colors they had never seen before.
The "Variable" Concept: The brain learns to say, "This spot is always a 'noun,' even if the noun changes." It treats a specific item as a placeholder (a variable) for a whole category.

Part 4: The "Variable" Model (The Ultimate Abstraction)

The Analogy: The Mad Libs Game
In the final part of the thesis, the author builds a model called HVM (Hierarchical Variable Model). Think of this as a super-smart "Mad Libs" generator.

How it works: The model doesn't just learn that "Red" comes after "Blue." It learns that "A Color" comes after "A Color." It replaces specific items with symbols (variables) that represent categories.
The Power: This allows the model to compress information massively. Instead of remembering 1,000 specific sentences, it remembers one rule: "Subject + Verb + Object."
The Comparison: The author tested this against Large Language Models (like the AI you are talking to now). The thesis claims that while big AI models are great at mimicking human speech, they often struggle to truly "abstract" in the way humans do. They memorize the specific words rather than learning the underlying variable rules that allow for true generalization to new situations.

Summary: The Journey from Dionysius to Apollo

The title references Greek mythology: Dionysus represents chaos, wine, and raw emotion, while Apollo represents order, logic, and structure.

The thesis argues that our brains are constantly turning the Dionysian chaos of sensory input into Apollonian order. We do this by:

Chunking: Grouping raw data into manageable pieces.
Composing: Stacking those pieces into complex structures.
Abstracting: Replacing specific pieces with general rules (variables) to understand the world at a deeper level.

This process allows us to move from simply reacting to noise to understanding the hidden laws of the universe, one pattern at a time.

Technical Summary: From Dionysius Emerges Apollo – Learning Patterns and Abstractions from Perceptual Sequences

1. Problem Statement

The central problem addressed in this dissertation is the gap between human and machine perception regarding the emergence of structured entities from chaotic sensory streams. While humans effortlessly parse continuous, high-dimensional sensory data into discrete, reusable, and hierarchical entities (e.g., recognizing "traffic signals" or grouping visual parts), current connectionist AI systems (such as Large Language Models and deep neural networks) struggle to learn structured symbols and factorize data without explicit supervision or human-defined priors.

The thesis posits that the cognitive mechanism of chunking—the tendency to segment sequences into recurring, cohesive units—is the fundamental computational principle that bridges this gap. The work investigates how chunking facilitates the transition from concrete perceptual sequences to abstract representations, enabling efficient memory compression, generalization, and the recursive construction of complex knowledge from simple primitives.

2. Methodology

The research employs a multi-faceted approach combining behavioral experiments, normative computational modeling, and algorithmic analysis. The methodology is structured across four primary projects:

A. Behavioral Experiments

Serial Reaction Time (SRT) Tasks: Participants performed tasks requiring them to press keys corresponding to visual cues. The underlying statistical structure of the sequences was manipulated (e.g., varying chunk lengths like AB or ABC) to observe how humans adapt their reaction times and error rates.
Instruction Manipulation: Experiments varied instructions to prioritize either speed or accuracy to test the trade-off in chunking behavior.
Sequence Memory and Recall: Participants memorized sequences of colors or abstract symbols. The study introduced two types of abstract motifs:
- Projectional Motifs: Patterns where superficial content is mapped to a shared abstract space (e.g., $GGGE$ and $FFFD$ both mapping to $XXXY$).
- Variable Motifs: Patterns containing invariant parts and a variable slot (e.g., $GGGX$ where $X$ varies).
- Transfer blocks tested whether learning these motifs facilitated recall of novel sequences sharing the same abstract structure.

B. Computational Modeling

The thesis develops a series of normative models that treat chunking not merely as a heuristic but as a rational strategy for an intelligent agent:

Rational Chunking Model: A model that learns chunks by optimizing a utility function that trades off speed and accuracy. It infers chunk boundaries based on sequential statistics and task demands.
Hierarchical Chunking Model (HCM): A generative and inverse recognition model that learns nested hierarchical structures. It parses sequences into atomic units, learns associative statistics between them, and recursively merges correlated chunks into larger composites. It utilizes a "dictionary" of learned chunks to factorize high-dimensional probability distributions.
Hierarchical Variable Model (HVM): An extension of HCM that integrates abstraction. It organizes learned chunks into a Trie structure for efficient retrieval and proposes symbolic variables to represent categories of chunks with similar interaction properties. This allows the model to discover invariant patterns and generalize to novel sequences by replacing specific instances with abstract variables.

C. Algorithmic Evaluation

The models were evaluated against:

Human behavioral data (reaction times, recall accuracy).
Baseline models (e.g., standard Recurrent Neural Networks, Bayesian ideal observers without chunking).
Large Language Models (LLMs) on sequence memory tasks to compare abstraction capabilities.
Unsupervised pattern discovery on real-world data, including text (from The Hunger Games) and resting-state fMRI data.

3. Key Contributions

1. Normative Justification for Chunking

The work provides a normative account for chunking, demonstrating that it is a rational solution to the speed-accuracy trade-off in sequence learning. The models show that humans adaptively learn longer chunks when speed is prioritized and shorter, more accurate chunks when precision is required, optimizing the trade-off between processing efficiency and error rates.

2. Hierarchical Representation Learning (HCM)

The thesis introduces HCM, a model that learns nested hierarchical structures from the ground up without pre-defined primitives.

Compositionality: HCM demonstrates how simple chunks can be recursively composed into complex entities, effectively solving the "symbol emergence" problem by learning primitives from data rather than assuming them.
Factorization: By grouping sequential units into chunks, the model factorizes high-dimensional sequence distributions (e.g., $P(x_1, x_2, \dots, x_n)$ ) into independent sub-distributions (e.g., $P(x_1, x_2)P(x_3, x_4)\dots$ ), reducing computational complexity.
Data Efficiency: HCM adapts its representation faster than RNNs on sequences with embedded hierarchies, requiring less data to learn the underlying structure.

3. Abstraction and Variable Learning (HVM)

The HVM extends chunking into the abstract domain by:

Symbolic Abstraction: Learning to replace recurring concrete chunks with abstract variables (symbols) that denote categories.
Memory Organization: Organizing the memory dictionary into a Trie structure based on common subparts, which reduces search steps for parsing and mimics efficient human memory retrieval.
Generalization: Showing that learning abstract motifs (projectional and variable) significantly improves transfer to novel sequences, a capability that standard chunking models and LLMs (in the tested context) fail to replicate effectively.

4. Empirical Validation

Human Alignment: The models' negative log-likelihood correlates strongly with human sequence recall times, outperforming models that do not learn chunks or variables.
Neuroscientific Application: The models were applied to fMRI data to identify nested network structures in the brain (e.g., regions for affect processing and visual attention), suggesting that neural activity exhibits hierarchical chunking.
Text Analysis: Applied to book text, the model successfully learned word parts, common phrases, and grammatical structures, mirroring human reading processes.

4. Results

Behavioral Findings: Humans dynamically adjust chunking strategies based on sequence statistics and task instructions. In memory tasks, participants who learned abstract motifs (both projectional and variable) showed superior recall and transfer performance compared to control groups, indicating that abstraction aids in compressing and generalizing sequence information.
Model Performance:
- The Rational Chunking Model successfully predicted human reaction time patterns under speed-accuracy trade-offs.
- HCM demonstrated superior data efficiency and interpretability compared to RNNs, successfully recovering the generative hierarchy of synthetic sequences.
- HVM outperformed LLMs in transfer tasks requiring the identification of variable structures, highlighting a limitation in current LLMs' ability to form explicit, reusable abstract representations from training sequences.
Unsupervised Discovery: The models successfully identified meaningful, nested structures in naturalistic data (text and brain imaging) without supervision, validating the hypothesis that chunking is a universal mechanism for discovering invariant patterns.

5. Significance and Claims

The thesis claims that chunking and abstraction are simple computational principles that give rise to structured knowledge acquisition. By learning to associate nearby elements and then abstracting these associations into reusable symbols, an agent can:

Bridge the AI-Human Gap: Provide a mechanism for connectionist systems to learn discrete, interpretable entities from continuous data, addressing issues of interpretability and generalization.
Enable Recursive Construction: Allow for the construction of highly complex structures from the ground up by reusing and composing previously learned primitives.
Explain Cognitive Phenomena: Offer a unified framework for understanding diverse cognitive phenomena, including memory organization (chunking in short-term memory), grammar acquisition, and the formation of abstract concepts.

The work concludes that while the proposed models rely on specific assumptions (e.g., hierarchical generative processes and greedy parsing strategies), they offer a plausible, computationally efficient, and cognitively grounded explanation for how structured representations emerge from perceptual sequences. The thesis calls for future research to explore the neural implementation of these principles and their application to higher-dimensional data domains.