Statistical Properties of the King Wen Sequence: An Anti-Habituation Structure That Does Not Improve Neural Network Training

1. Problem Statement

The paper investigates a hypothesis linking ancient combinatorial structures to modern machine learning optimization. Specifically, it asks whether the King Wen sequence of the I-Ching (c. 1000 BC)—a 3,000-year-old ordering of 64 hexagrams (6-bit binary vectors)—possesses statistical properties (such as "anti-habituation" or information-theoretic surprise) that could improve neural network training.

The core intuition is that the sequence's unique ordering, which avoids simple repetition and maintains local balance, might function similarly to curriculum learning or curiosity-driven exploration, potentially helping models escape local minima or learn more efficiently than random or sequential ordering.

2. Methodology

The research employs a two-phase approach: rigorous statistical characterization followed by empirical machine learning experiments.

A. Statistical Characterization (Monte Carlo Analysis)

• Baseline: The authors compared the King Wen sequence against 100,000 random permutations of the 64 hexagrams.
• Metrics: They analyzed transition distances (Hamming distance), lag-1 autocorrelation, local yang/yin balance, and pair-wise asymmetry.
• Surprise Modeling: They defined "information-theoretic surprise" based on pattern similarity, incorporating traditional line-position weights and nuclear hexagram components.

B. Experimental Evaluation

The authors tested the hypothesis using Karpathy's autoresearch framework with a small GPT model (~11.5M parameters) trained for a fixed 5-minute budget.

• Hardware: Two platforms were used to isolate framework artifacts:
  1. NVIDIA RTX 2060 (PyTorch with torch.compile).
  2. Apple Silicon (MLX framework).
• Metric: Validation bits per byte (val_bpb), where lower is better.
• Experiments:
  1. Learning Rate (LR) Modulation: The King Wen surprise profile was used to multiplicatively modulate the learning rate schedule.
  2. Curriculum Ordering: Training batches were reordered based on the King Wen sequence's surprise rank, mapped against difficulty metrics (token diversity or compression ratio).
  3. Seed Sensitivity: A 30-seed sweep was conducted to distinguish genuine effects from natural variance.

3. Key Contributions

1. Statistical Confirmation: The paper provides the first rigorous statistical proof that the King Wen sequence is neither random nor algebraically systematic. It confirms four distinctive properties against 100,000 random baselines.
2. Negative Empirical Results: It demonstrates that these "interesting" statistical properties do not translate to performance gains in neural network training; in fact, they actively degrade performance.
3. Methodological Insight: The study isolates a significant confound where torch.compile (PyTorch) creates kernel-level optimizations that overfit to sequential data ordering, masking true learning dynamics.
4. Mechanistic Explanation: It offers a theoretical explanation for why "anti-habituation" in a fixed sequence fails in gradient-based optimization.

4. Key Results

A. Statistical Properties of the King Wen Sequence

The sequence exhibits four statistically significant properties (p-values < 0.05):

1. High Transition Distance: Mean Hamming distance between consecutive hexagrams is higher than random (98.2nd percentile).
2. Negative Lag-1 Autocorrelation: A large transition is systematically followed by a small one (p=0.037), creating an oscillating pattern of change.
3. Local Yang Balance: Groups of four consecutive hexagrams maintain perfect yin-yang balance significantly more often than chance (p=0.002).
4. Pair Asymmetry: Transitions within pairs are maximally different (complementary), while transitions between pairs are smoother.

B. Experimental Outcomes

• LR Modulation: Applying the King Wen surprise profile to the learning rate degraded performance at all tested amplitudes (0.15, 0.3, 0.5). The degradation increased monotonically with amplitude.
• Curriculum Ordering:
  • CUDA (PyTorch): King Wen ordering was the worst non-sequential ordering (val_bpb: 1.662 vs. Random: 1.614). However, random shuffling also outperformed sequential ordering, suggesting the benefit came from breaking torch.compile's sequential optimization artifacts rather than the curriculum itself.
  • MLX (Apple Silicon): No ordering showed a statistically significant improvement over sequential training. All results fell within the noise floor (±0.060 bpb).
• Seed Sensitivity: A 30-seed sweep established a natural variance range of 0.041 bpb. The "improvements" seen in random controls were within this noise, but the King Wen degradation (+0.032) exceeded the upper bound of natural variance, confirming the negative result is statistically meaningful.

5. Significance and Discussion

Why Anti-Habituation Failed

The authors argue that the very properties making the King Wen sequence combinatorially unique are counterproductive for gradient-based optimization:

• Excessive Variance: The high variance in the sequence causes larger effective perturbations in the learning rate than random noise, destabilizing gradient updates.
• Disrupted Momentum: The negative autocorrelation (oscillating intensity) prevents the optimizer from building consistent momentum, which is crucial for convergence.
• Lack of Adaptivity: Fixed sequences cannot adapt to the model's current loss landscape. Effective curriculum learning requires dynamic adjustment to the learner's state, which a static 3,000-year-old ordering cannot provide.

Broader Implications

• For I-Ching Studies: Confirms the sequence has genuine, non-random mathematical structure independent of ML applications.
• For Curriculum Learning: Highlights the gap between "combinatorial elegance" and "optimization utility." Fixed orderings, even those with theoretically appealing statistical profiles, do not substitute for adaptive curricula.
• For Experimental Rigor: The paper serves as a cautionary tale regarding framework artifacts. The discrepancy between CUDA and MLX results revealed that torch.compile can create false positives in curriculum experiments. It emphasizes the need for cross-platform validation and large seed sweeps to distinguish signal from noise.

Conclusion

The paper concludes that while the King Wen sequence is a fascinating combinatorial object with distinct statistical signatures, these properties do not confer an advantage in training neural networks. In fact, the sequence's high variance and oscillating nature actively hinder gradient-based optimization. The authors publish these negative results to close a speculative research direction and provide a robust methodological template for testing similar hypotheses.