Hebbian-Oscillatory Co-Learning

This paper introduces Hebbian-Oscillatory Co-Learning (HOC-L), a unified two-timescale framework that integrates hyperbolic sparse geometry with Kuramoto-based phase synchronization to enable connectivity consolidation only when phase coherence signals meaningful patterns, thereby achieving provable convergence and $O(n \cdot k)$ complexity in bio-inspired neural architectures.

The Gist

Imagine you are trying to teach a massive, chaotic city of 10,000 people (neurons) how to work together to solve a complex problem.

In most current computer brains (AI), the city is built with a fixed map. The roads (connections) are decided by engineers before the city opens, and they only change very slowly, if at all. The people talk to each other constantly, but they don't really know when to listen or who to listen to.

Hebbian-Oscillatory Co-Learning (HOC-L) is a new, smarter way to build this city. It combines two ancient, biological rules of how brains actually work into one unified system.

Here is the simple breakdown using a creative analogy:

1. The Two Rules of the Brain

To understand HOC-L, you need to know the two rules it combines:

• Rule A: "Practice Makes Permanent" (Hebbian Plasticity)

  • The Analogy: Imagine two neighbors who keep talking to each other every day. Eventually, they build a private, paved driveway between their houses so they can talk faster. If they stop talking, the driveway gets overgrown and disappears.
  • In AI: If two neurons fire together often, the connection between them gets stronger. If they don't, the connection weakens. This is how the brain "rewires" itself over days and weeks.

• Rule B: "The Dance Floor Effect" (Oscillatory Synchronization)

  • The Analogy: Imagine a huge dance party. At first, everyone is dancing to their own beat, totally out of sync. But suddenly, a DJ drops a beat, and everyone starts clapping in rhythm. When everyone is clapping together (synchronized), they can hear each other clearly and pass messages easily. When they are out of sync, it's just noise.
  • In AI: Neurons oscillate (vibrate) at certain speeds. When they vibrate in sync, they can communicate effectively. When they are out of sync, they ignore each other.

2. The Problem with Current AI

Current AI tries to do these two things separately.

• It tries to build the roads (structure) using math, but it doesn't listen to the "dance rhythm."
• It tries to make people dance (synchronize), but it doesn't let them build new roads based on that dancing.

It's like trying to build a city where the roads are fixed, but the people are told to dance. They can't build better roads because they are dancing together.

3. The HOC-L Solution: The "Synchronization Gate"

HOC-L introduces a brilliant "Gatekeeper" that links the two rules.

The Scenario:
Imagine the city has a giant Order Parameter (let's call it the "Vibe Meter").

• Low Vibe (0): Everyone is dancing to different songs. Chaos.
• High Vibe (1): Everyone is dancing in perfect unison. Harmony.

The Magic Mechanism:
HOC-L says: "We will only build new roads (strengthen connections) when the Vibe Meter is high."

1. Fast Time (The Dance): The neurons dance and try to sync up. This happens very fast (milliseconds).
2. The Check: The system checks the "Vibe Meter."
   • Is the group synchronized? No? Then, STOP! Do not build any new roads. The connections stay weak.
   • Is the group synchronized? Yes! GO! Now, the neurons that are dancing together get to build a permanent, strong road between them.

4. Why is this a Big Deal?

• It's Efficient (The Sparse City):
  In a normal city, everyone tries to build a road to everyone else. That's a traffic nightmare (too much math). In HOC-L, roads are only built between people who are actually dancing together. This keeps the city small, fast, and energy-efficient. It's like having a city where you only pave the streets you actually use.

• It Learns Patterns:
  Because the system only strengthens connections when the group is in sync, it naturally discovers "cliques" or "teams." If a group of neurons is good at solving math problems, they will sync up, build strong roads, and become a dedicated "Math Team." If another group is good at recognizing faces, they sync up and become a "Face Team." The structure of the brain emerges from the behavior.

• It's Stable:
  The paper proves mathematically that this system won't go crazy. It will eventually settle into a stable state where the roads are perfectly laid out for the specific tasks the city needs to do.

Summary in One Sentence

HOC-L is a smart learning system that says, "We only build permanent connections between neurons when they are dancing in perfect harmony," allowing the brain to rewire itself efficiently and naturally, just like a human brain does.

It turns the chaotic noise of a million neurons into a synchronized orchestra, where the music itself decides which instruments get to play together forever.

Technical Summary

Here is a detailed technical summary of the paper "Hebbian-Oscillatory Co-Learning (HOC-L)" by Hasi Hays.

1. Problem Statement

Current artificial neural networks (ANNs) typically treat structural learning (changing connectivity) and dynamic coordination (information flow) as separate processes.

• Standard Deep Learning: Optimizes weights on a fixed topology via gradient descent, lacking mechanisms for activity-dependent structural rewiring.
• Sparse Methods (e.g., Lottery Ticket Hypothesis): Address structural sparsity but lack oscillatory coordination mechanisms.
• Oscillator Models: Capture temporal coordination (e.g., Kuramoto models) but usually operate on fixed graphs without plasticity.

Biological evidence suggests these processes are coupled: oscillatory coherence (phase synchronization) modulates synaptic plasticity (Hebbian learning). Neurons that fire together (synchronized) wire together, but this consolidation only occurs when phase coherence signals a meaningful pattern. HOC-L aims to unify these mechanisms into a single framework that mimics this biological interplay.

2. Methodology: The HOC-L Framework

HOC-L is a unified, two-timescale dynamical system that couples Resonant Sparse Geometry Networks (RSGN) and Selective Synchronization Attention (SSA).

A. Core Components

1. Structural Backbone (RSGN):

   • Embeds neural representations in a Poincaré ball (hyperbolic space).
   • Uses geodesic distance to define a sparse computational graph. Nodes $i$ and $j$ connect only if their hyperbolic distance $d_H(z_i, z_j) < \delta$.
   • This provides input-dependent sparsity ($k \ll n$).

2. Dynamic Coordination (SSA):

   • Replaces standard dot-product attention with Kuramoto oscillator dynamics.
   • Each unit $i$ has an intrinsic frequency $\omega_i$ and phase $\theta_i(t)$.
   • Attention weights emerge from phase-locking: units synchronize if their phases align.
   • A macroscopic order parameter $r(t)$ measures global coherence ($r \approx 0$ is incoherent, $r \approx 1$ is fully synchronized).

3. The Coupling Mechanism: Synchronization-Gated Plasticity

   • This is the novel core of HOC-L. Structural updates (Hebbian learning) are gated by the oscillator ensemble's coherence.
   • Rule: $\Delta W_{ij} = \eta \cdot x_i x_j \cdot G(r(t))$, where $G(r)$ is a smooth sigmoid gate (approximating an indicator function).
   • Logic: Structural connections are only strengthened (consolidated) when $r(t) > r_c$ (a critical threshold). If the network is incoherent, plasticity is suppressed, preventing spurious learning.

B. Two-Timescale Dynamics

The system operates on two distinct timescales:

• Fast Timescale ($\tau_{fast}$): Oscillator phases evolve rapidly toward synchronization, and task-specific gradient updates occur.
• Slow Timescale ($\tau_{slow}$): Hebbian structural updates modify the weight matrix $W$ and graph topology.
• Separation: $\tau_{fast} \ll \tau_{slow}$ ensures the oscillatory dynamics reach a quasi-stationary state before structural changes occur.

3. Key Contributions

1. Unified Dynamical System: Formulated the first system coupling Kuramoto-type phase synchronization with synchronization-gated Hebbian structural plasticity, grounded in hyperbolic geometry.
2. Theoretical Convergence: Proved that the joint system converges to a stable equilibrium using a composite Lyapunov function ($V(W, \theta)$). The proof establishes explicit conditions for the timescale separation ratio ($\tau_{fast}/\tau_{slow}$) required for stability.
3. Complexity Efficiency: Derived an architecture with $O(n \cdot k)$ computational complexity (where $k \ll n$ is the sparse neighborhood size), preserving the efficiency of both parent frameworks and avoiding the $O(n^2)$ cost of standard self-attention.
4. Experimental Protocols: Proposed comprehensive benchmarks (synthetic, graph classification, long-range sequences, neuromorphic) and ablation studies to validate the theory.

4. Results and Simulations

The paper presents numerical simulations (not full-scale training on ImageNet/LLMs yet, but theoretical validation via simulation) confirming the framework's behavior:

• Timescale Separation: Simulations show distinct frequency content between fast phase updates (rapid oscillations decaying as synchronization occurs) and slow weight updates (modulated by the gate).
• Emergent Structure: In a system with two frequency clusters, the model autonomously discovered the cluster structure. Strong connections formed only between synchronized oscillators, while desynchronized groups remained unconnected. This resulted in a block-diagonal weight matrix, mirroring modular connectivity in biological microcircuits.
• Lyapunov Stability: Trajectories in the $(W, \theta)$ space demonstrated monotonic decrease of the composite Lyapunov function, converging to a stable basin where oscillators are synchronized and weights are regularized.
• Gate Mechanism: The plasticity gate effectively suppressed weight updates during incoherent states ($r < r_c$) and activated them only when the network identified a coherent computational pattern.

5. Significance and Implications

• Biological Plausibility: HOC-L bridges a gap between neuroscience and deep learning by implementing "communication through coherence." It models how the brain uses oscillatory rhythms to gate long-term structural changes (LTP/LTD).
• Efficiency: By combining hyperbolic sparsity with oscillatory gating, the model achieves high efficiency ($O(n \cdot k)$), making it suitable for long-range sequence modeling and large graphs where standard attention is prohibitive.
• Neuromorphic Potential: The architecture is naturally suited for neuromorphic hardware. The fast oscillatory dynamics can map to analog oscillators, while the slow Hebbian updates can be implemented in memristive crossbar arrays, potentially offering significant energy efficiency gains over GPU-based training.
• Generalization: The framework generalizes existing approaches; setting specific parameters (e.g., $K=0$ or $\eta=0$) reduces HOC-L to RSGN, SSA, or standard sparse networks, suggesting it is a unifying paradigm rather than a replacement.

In summary, HOC-L proposes that structure and dynamics are deeply coupled. By using oscillatory coherence as a gate for structural plasticity, the model achieves a self-organizing, sparse, and stable neural architecture that aligns with biological principles of learning.