Exploring the holographic entropy cone via reinforcement learning

This paper introduces a reinforcement learning algorithm to explore the holographic entropy cone by searching for graph realizations of target entropy vectors, successfully rediscovering known properties for N=3 and resolving the status of six "mystery" extreme rays for N=6 by proving three are realizable while suggesting the other three reveal unknown holographic inequalities.

The Gist

The Big Picture: Mapping a Hidden Shape

Imagine the universe of quantum information is a giant, multi-dimensional room filled with invisible shapes. Physicists are trying to map out the boundaries of a specific shape called the Holographic Entropy Cone (HEC).

Think of this shape like a giant, complex crystal. Inside this crystal, certain patterns of "entropy" (a measure of disorder or information) are allowed to exist. Outside the crystal, those patterns are impossible. The goal of this paper is to figure out exactly where the walls of this crystal are and what its sharp corners look like.

For small, simple crystals (with 3 parties), physicists already knew the shape. But for a larger, more complex crystal (with 6 parties), the shape is so complicated that traditional math tools get stuck. It's like trying to find the edge of a massive, foggy mountain range by walking blindly; you might bump into a wall, but you won't know if it's the only wall or if there are others hidden in the mist.

The New Tool: A Digital "Sniffer Dog"

To solve this, the authors built a Reinforcement Learning (RL) algorithm. You can think of this algorithm as a highly trained digital sniffer dog.

Here is how the dog works:

1. The Target: The researchers give the dog a specific "scent" (a target entropy vector). This scent represents a pattern they want to see if it exists inside the crystal.
2. The Search: The dog tries to build a "graph" (a network of connected dots and lines with weights) that produces that exact scent.
3. The Reward:
   • If the dog builds a graph that matches the scent perfectly, it gets a perfect score (100%). This means the scent is inside the crystal.
   • If the scent is outside the crystal (impossible), the dog can't get a perfect score. Instead, it builds the graph that comes closest to the scent. It gets a lower score, but that score tells the researchers how far away the scent is from the crystal's wall.

The Two Main Discoveries

1. The "Training Wheels" Test (N=3)

First, the team tested their dog on a small, simple crystal (3 parties) where they already knew the rules.

• The Test: They gave the dog a scent that they knew was outside the crystal because it broke a known rule called "Monogamy of Mutual Information" (MMI).
• The Result: The dog didn't just say "No." It started walking in a specific direction, guided by the "reward gradient" (a mathematical compass). It walked straight toward the invisible wall of the crystal.
• The Magic: When the dog hit the wall, the direction it was walking pointed exactly perpendicular to the wall. By looking at that direction, the dog effectively rediscovered the rule (MMI) that defines that wall, even though the researchers told it to pretend it didn't know the rule. This proved the dog could find the edges of the shape just by trying to get a high score.

2. Solving the "Mystery Rays" (N=6)

Next, they moved to the big, complex crystal (6 parties). In a previous study, physicists found 208 "extreme rays" (sharp corners of the crystal). They could prove 150 of these corners existed inside the crystal, and 52 were definitely outside. But there were 6 "Mystery Rays" that were stuck in limbo. They didn't break any known rules, but no one could find a graph to build them.

• The Investigation: The team sent their RL dog to hunt for graphs for these 6 mystery rays.
• The Breakthrough:
  • The dog successfully found graph realizations for 3 of the 6 rays. This proved these 3 rays are genuine corners of the holographic crystal.
  • For the other 3 rays, the dog tried very hard but failed to find a graph, even after trying many different sizes of networks.
  • The Conclusion: The authors suspect these last 3 rays are not real. They are surrounded by other rays that are definitely outside the crystal. This suggests that there are hidden rules (new inequalities) that we don't know about yet, which are keeping these 3 rays outside the crystal.

The Takeaway

This paper is a success story of using machine learning as a discovery tool. Instead of just crunching numbers to solve a puzzle, the authors used an AI to "feel" its way through a high-dimensional space.

• They proved the AI can find the boundaries of a complex shape.
• They used the AI to solve a specific mystery: confirming that 3 "mystery" corners of the holographic universe are real.
• They provided strong evidence that the other 3 mystery corners are fake, implying that physicists need to discover new laws of physics (new entropy inequalities) to explain why they don't exist.

In short, they built a digital explorer that helped map the edges of a shape that was previously too foggy to see clearly.

Technical Summary

Technical Summary: Exploring the Holographic Entropy Cone via Reinforcement Learning

Problem Statement
The holographic entropy cone (HEC) characterizes the set of all entropy vectors realizable by holographic quantum states, which are constrained by specific holographic entropy inequalities (HEIs) beyond the standard quantum entropy inequalities (Subadditivity and Strong Subadditivity). While the HEC is fully characterized for $N \le 5$ parties, the case for $N \ge 6$ remains an open problem due to the doubly exponential growth of the search space for extreme rays and facets. A specific challenge identified in prior work [1] involves 208 new classes of extreme rays of the Subadditivity Cone (SAC) for $N=6$. Of these, 6 "mystery rays" satisfy all known HEIs but lacked confirmed graph realizations, leaving their status as genuine extreme rays of the HEC undetermined. Traditional combinatorial search methods and analytical approaches become computationally intractable for $N=6$.

Methodology
The authors develop a Reinforcement Learning (RL) algorithm to address the problem of entropy vector realizability. The core task is formulated as a search problem: given a target entropy vector $\vec{S}_{\text{target}}$, find a weighted graph $G$ whose min-cut entropies match $\vec{S}_{\text{target}}$.

• RL Framework: The algorithm treats the graph configuration (edge weights of a complete graph with $N+1$ boundary vertices and $n$ internal vertices) as the state. The policy network (a feedforward neural network) maps the current graph state to actions (updates to edge weights).
• Reward Function: The reward $R$ is defined as the cosine similarity between the target entropy vector and the entropy vector $\vec{S}(G)$ induced by the graph's min-cuts:
  $$R = \frac{\vec{S}_{\text{target}} \cdot \vec{S}(G)}{\|\vec{S}_{\text{target}}\| \|\vec{S}(G)\|}$$
  Since the HEC is a cone, a reward of $R=1$ implies the target is realizable (lies inside the HEC). If the target is outside the HEC, the algorithm seeks the graph that maximizes $R$, effectively finding the closest point on the cone boundary.
• Gradient-Based Navigation: For targets outside the HEC, the gradient of the reward function with respect to the target vector points toward the nearest boundary facet. The authors utilize this property to navigate from non-realizable vectors toward the HEC boundary, potentially identifying unknown constraints.
• Validation and Certification: While the RL algorithm operates with finite numerical precision, any candidate graph output is analytically verified. Edge weights are rescaled to rational or integer values, and min-cuts are recomputed exactly to confirm realizability.

Key Contributions and Results

1. Proof of Concept ($N=3$):

   • The authors applied the algorithm to the $N=3$ case, where the HEC is fully specified by Subadditivity (SA) and the Monogamy of Mutual Information (MMI).
   • Analytical Validation: They derived closed-form expressions for the reward landscape on the $S_3$-symmetric slice. The RL results showed a Pearson correlation of 0.996 with analytical predictions, validating the algorithm's ability to recover the reward landscape.
   • Gradient Rediscovery: Starting from an extreme ray of the SAC that violates MMI, the algorithm successfully navigated toward the HEC boundary. The gradient direction aligned with the normal vector of the MMI facet, effectively "rediscovering" the MMI inequality without prior knowledge of it, demonstrating the utility of the reward gradient as a navigator for unknown facets.

2. Resolution of $N=6$ Mystery Rays:

   • The algorithm was applied to the 6 "mystery rays" of the $N=6$ SAC identified in [1].
   • Realizable Rays: The algorithm successfully found graph realizations for 3 of the 6 rays (specifically rays 146, 180, and 181). These realizations involved graphs with up to 13 internal vertices (20 total vertices). The authors provided explicit integer-weighted graph constructions for these rays, proving they are genuine extreme rays of the $N=6$ HEC.
   • Non-Realizable Candidates: For the remaining 3 rays (110, 145, and 168), the algorithm failed to find realizations despite extensive training across various internal vertex counts ($n=2$ to $13$).
   • Evidence for New Inequalities: By plotting the maximum reward achieved for all 208 SAC extreme rays, the authors observed that the 3 unresolved mystery rays are surrounded by non-realizable rays (those violating known HEIs), whereas the 3 resolved rays are surrounded by realizable ones. This suggests the remaining 3 rays are likely not realizable, implying the existence of unknown holographic entropy inequalities for $N=6$.

Significance and Claims
The paper claims that reinforcement learning provides a powerful, systematic tool for exploring the holographic entropy cone, particularly in regimes where combinatorial search is intractable.

• Classification Capability: The algorithm serves as a robust classifier, distinguishing between realizable and non-realizable entropy vectors by the magnitude of the achieved reward.
• Facet Discovery: The gradient of the reward function acts as a navigator toward unknown facets of the cone. The successful rediscovery of the MMI inequality in the $N=3$ case demonstrates the potential for this method to discover new holographic inequalities in higher dimensions.
• Resolution of Open Problems: The work resolves the status of 3 of the 6 mystery rays for $N=6$, confirming them as extreme rays of the HEC, and provides strong evidence that the other 3 are not, thereby narrowing the search space for new HEIs.

The authors note that while the current implementation uses a vanilla policy gradient algorithm, the results suggest that more sophisticated RL approaches (e.g., Proximal Policy Optimization) or hybrid algorithms could further improve efficiency and stability, particularly for the high-dimensional $N=6$ entropy space ($D=63$). The paper does not claim to have fully characterized the $N=6$ HEC but rather provides a method to systematically probe its structure and resolve specific open questions regarding extreme rays.