🤖 machine learning

Compositionality Emerges in a Narrow Depth-Connectivity Regime: Architecture Constraints and Solution Manifolds

This paper demonstrates that compositional internal structures in neural networks emerge only within a narrow, target-dependent regime of specific connectivity patterns and network depths, and proposes similarity-based pruning and a depth predictor to identify and exploit these conditions while providing a theoretical framework explaining why gradient descent otherwise converges to fractured solutions.

Original authors: Dat H. Do, Rushi Shah, Duc V. Le, Dianbo Liu

Published 2026-06-19

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Dat H. Do, Rushi Shah, Duc V. Le, Dianbo Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Finding the "Goldilocks" Zone for AI Brains

Imagine you are trying to teach a robot to draw a picture of a skull. You want the robot to learn the concept of a "skull" by combining simple, reusable parts: an eye socket, a jaw, a nose. This is called compositionality. It's like having a set of Lego bricks where you can snap the same "eye" piece onto different models to build a car, a house, or a robot.

However, most modern AI models (trained with standard methods) don't learn this way. Instead of building with clean Lego bricks, they tend to build with a giant, tangled ball of yarn. Every part of the image is mixed together with every other part. If you tweak one tiny thread in the yarn, the whole picture might turn into a mess. This is called fractured entanglement.

The authors of this paper asked a simple question: Is there a specific way to build the robot's brain so that it naturally learns to use clean, reusable Lego bricks instead of tangled yarn?

Their answer is a surprising "Yes," but with a very strict catch: The brain must be built in a very specific "Goldilocks" zone of size and depth.

The Two Rules for Success

The paper discovers that for an AI to learn these clean, reusable parts, two things must happen simultaneously. If you get either one wrong, the AI fails.

1. The Connectivity Rule: "Less is More, But Only If It's the Right Less"

Imagine you have a room full of people (neurons) trying to solve a puzzle.

The Mistake: If you let everyone talk to everyone else (a fully connected, dense network), they all start shouting over each other. They get confused, and no one takes responsibility for a specific piece of the puzzle.
The Fix: You need to cut the connections. But you can't just cut random ones. You have to cut the exact right ones.
The Analogy: Think of a busy highway. If you randomly close lanes, traffic jams get worse. But if you close the specific lanes that are causing the gridlock and keep the ones that allow cars to flow smoothly, traffic clears up.
The Paper's Discovery: The authors found a special pruning method (cutting connections) called Similarity-based Pruning (SP). This method looks at which neurons are doing the exact same job and cuts the duplicates. This forces the remaining neurons to specialize. One neuron becomes the "eye" expert, another becomes the "jaw" expert.

2. The Depth Rule: "Not Too Shallow, Not Too Deep"

Now, imagine the layers of the brain as floors in a building.

Too Shallow (1-2 floors): The building is too short. The "eye" expert and the "jaw" expert are on the same floor and can't organize their work properly. They can't build a complex structure.
Too Deep (100 floors): The building is too tall. The instructions get lost in the elevator shaft. By the time the signal reaches the top, the "eye" and "jaw" experts have forgotten who they are supposed to be, and they start mixing up their jobs again.
The Sweet Spot: There is a specific number of floors (depth) that is perfect for the specific picture you are trying to draw.
- For a simple apple, maybe 12 floors is perfect.
- For a complex butterfly, maybe 14 floors is perfect.
- If you add or remove just one floor, the magic disappears, and the AI goes back to building with tangled yarn.

The "Sweet Spot" Regime

The paper calls this the Narrow Depth–Connectivity Regime.

Think of it like tuning a radio.

Connectivity is the antenna. It needs to be the right shape (sparse and specific) to catch the signal.
Depth is the frequency dial. It needs to be turned to the exact number.
If you have the right antenna but the wrong frequency, you hear static.
If you have the right frequency but the wrong antenna, you hear nothing.
Only when both are perfect does the music (compositionality) play clearly.

How They Proved It

The researchers didn't just guess; they built a testing ground called EMC2-Bench.

The Test: They trained AI models to draw pictures (like skulls, butterflies, and apples).
The "Wiggle" Test: After training, they took a tiny "wiggle" (a small random change) to a single number inside the AI's brain.
- If the AI was using tangled yarn, that tiny wiggle would break the whole picture (e.g., the skull turns into a blob).
- If the AI was using Lego bricks, that tiny wiggle would only affect one specific part (e.g., the eye gets slightly bigger, but the jaw stays perfect).
The Result: They found that standard training almost always produced tangled yarn. But when they used their special pruning method (SP) and tuned the depth to the exact "sweet spot," the AI suddenly started building with clean Lego bricks.

Why Does This Happen? (The Theory)

The paper offers a mathematical explanation using a concept called Volume Ratio.

Imagine a giant ocean of all possible ways to build a brain.

The Tangled Ocean: Most of the ocean is filled with "tangled" solutions. They are huge, easy to find, and easy to fall into.
The Compositional Island: The "clean Lego" solutions are like a tiny, hidden island in the middle of that ocean. It is very small and hard to find.

The paper argues that:

If your brain is too wide (too many neurons), the "tangled" ocean gets bigger, and the "clean" island gets relatively smaller. You are statistically likely to get lost in the tangled mess.
If your brain is too deep or too shallow, the island disappears entirely.
The Magic: If you hit the exact right width and depth, the "tangled" ocean shrinks, and the "clean" island becomes the only place left to stand. The AI has to find the clean solution because there is nowhere else to go.

Summary

This paper shows that AI models don't naturally learn to think in reusable, modular parts. They naturally learn to be messy and tangled.

However, if you force the AI into a very specific architectural box—where you cut the right connections and build the exact right number of layers—the AI is forced to organize itself. It stops being a tangled ball of yarn and starts building with clean, reusable Lego bricks. This is the secret to making AI that can generalize and understand the world more like a human does.

Technical Summary: Compositionality Emerges in a Narrow Depth–Connectivity Regime

Problem Statement

Modern foundation models (LLMs, VLMs, diffusion models) exhibit strong capabilities but suffer from fragile generalization, particularly in compositional understanding. Small, task-irrelevant perturbations often trigger sharp behavioral shifts, suggesting a lack of robust internal compositional structure. While evolutionary algorithms (e.g., NEAT on Picbreeder) can discover "Unified Factored Representations" (UFR)—unified, reusable, and semantically stable primitives—standard gradient-based optimization (SGD) typically converges to "Fractured Entangled Representations" (FER). In FER, concepts are distributed across overlapping neurons, and features are polysemantic, failing to align with stable, interpretable meanings.

The central question addressed is: Can monosemanticity and compositionality emerge during standard gradient-based backpropagation, and if so, under what specific architectural conditions? The authors hypothesize that this emergence is not a default outcome of scale or low loss but is governed by a structural reachability problem defined by the interplay of network depth and connectivity.

Methodology

1. Evaluation Suite: EMC2-Bench

To quantify compositionality, the authors introduce EMC2-Bench, a perturbation-based evaluation suite.

Metric: The "Compositional Score" is defined as the fraction of non-zero parameters that, when perturbed by noise ( $\delta$ ), preserve the global semantic identity of the generated image.
Procedure: For each non-zero weight $w$ , a perturbed copy $w' = w + \delta$ is created. The network generates an image, which is evaluated by a Vision-Language Model (VLM) judge (Qwen3-VL-4B-Thinking) to determine if the semantic content (e.g., "skull") remains intact.
Baseline: The benchmark compares dense MLPs trained via SGD against evolutionary solutions (Picbreeder CPPNs) and various pruning strategies.

2. Structural Interventions

Similarity-based Pruning (SP): To recover compositional connectivity, the authors propose an iterative prune-retrain algorithm.
- Mechanism: Within each layer, neuron-wise feature maps are extracted. Neurons with high cosine similarity are identified as redundant. One neuron from each redundant pair is randomly selected for removal, and its incoming/outgoing weights are zeroed.
- Goal: This forces the network to allocate distinct neurons to distinct semantic factors, collapsing entangled solutions into sparse, reusable circuits.
Heuristic Depth Predictor: Recognizing that optimal depth is target-dependent, the authors propose using the PNG compression ratio of the target image as a proxy for image complexity. A linear relationship is observed between image complexity and the empirically optimal depth, allowing for a lightweight prediction of the "sweet spot" depth without exhaustive sweeping.

3. Theoretical Framework

The paper provides a theoretical justification based on:

Compositional Sparsity: Drawing on the manifold hypothesis, the authors argue that compositional solutions factorize through a Directed Acyclic Graph (DAG) with bounded local arity.
Volume-Ratio Argument: They define the "compositional volume ratio" $\rho(W, L)$ as the probability of a random parameter configuration landing in a compositional basin. They prove that $\rho$ is sharply concentrated at a specific $(W_{min}, L^*)$ and decays exponentially as width ( $W$ ) or depth ( $L$ ) deviates from this point.
Feature Interference Bounds: Using the Welch bound, they demonstrate that compositional sparsity reduces the geometric lower bound on feature interference, making monosemantic (orthogonal) features geometrically feasible only when the number of primitives $r$ is less than or equal to the dimension $d$ .

Key Results

1. The Narrow Depth–Connectivity Regime

Compositionality does not emerge universally; it appears only in a specific regime:

Connectivity: Compositionality is heavily dependent on which connections remain, not just the sparsity level. Standard pruning methods (Lottery Ticket Hypothesis, Wanda, LLM-Pruner) fail to recover compositional structures (scoring 0 on EMC2-Bench). Only Similarity-based Pruning (SP) yields non-zero scores, indicating that the organization of connections is critical.
Depth: Compositionality exhibits a non-monotonic relationship with depth. For specific targets (e.g., skull, butterfly, apple), the compositional score peaks at a specific depth and drops rapidly if the network is made shallower or deeper. This optimal depth is target-specific.

2. Optimization and Emergence

Optimizer Sensitivity: Even with the correct architecture, the optimizer matters. Muon (an optimizer utilizing Newton-Schulz iterations) achieves lower training loss and higher compositional scores than Adam. This suggests that better minima (flatter, more stable) are required to reach the compositional basin.
Fragility of FER: When depth or connectivity conditions are violated, gradient descent silently converges to fractured solutions. Small perturbations to weights in these solutions cause semantic drift, whereas compositional solutions remain robust.

3. Generalization to Unknown Targets

The method transfers to images where the underlying compositional solution is unknown (e.g., natural scenes, multi-object images). SP combined with the heuristic depth predictor successfully recovers monosemantic intermediate features and meaningful output variations under weight sweeping, though the effect is weaker for complex multi-object scenes.

Significance and Claims

The paper claims that generalization requires architectural constraints that make compositional solutions accessible to gradient-based training.

Refutation of Scale-Only Solutions: The authors argue that scaling alone (increasing width or depth) tends to yield fractured solutions because the relative measure of the compositional manifold shrinks as the parameter space grows (Volume-Ratio Argument).
Mechanistic Insight: The work bridges the gap between the existence of compact, modular implementations (proven by evolutionary algorithms) and the failure of vanilla backpropagation to find them. It posits that SGD is biased toward compositional basins only when the architecture is "just right" (narrow depth-connectivity sweet spot) and the optimizer is capable of finding flat minima.
Practical Implications: The introduction of SP and the depth predictor offers a pathway to induce compositional structure in standard MLPs without changing the training objective, simply by constraining the architecture and pruning strategy.

The authors remain modest, noting that while the framework explains single-object compositionality well, spatial compositionality in multi-object scenes remains an open boundary due to the exponential increase in required primitives ( $P$ ), which collapses the volume ratio $\rho$ . They do not claim to have solved general compositional reasoning for all vision-language tasks but have identified a specific structural regime where it is empirically and theoretically reachable.