🤖 AI

AI Learning and Conceptual Transfer in the Game of Hidden Rules

This report details a study on the Game of Hidden Rules (GOHR) where Transformer-based A2C agents are trained to infer hidden rules through trial-and-error feedback, examining the impact of feature-centric versus object-centric representations, rule difficulty, transfer learning, and generalization, while also analyzing human learning patterns assisted by pseudo-bots.

Original authors: Christo Mathew, Wentian Wang, Jacob Feldman, Lazaros K. Gallos, Paul B. Kantor, Vladimir Menkov, Hao Wang

Published 2026-08-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Christo Mathew, Wentian Wang, Jacob Feldman, Lazaros K. Gallos, Paul B. Kantor, Vladimir Menkov, Hao Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast landscape of artificial intelligence, researchers are constantly trying to teach machines how to learn the way humans do: by observing patterns, making mistakes, and figuring out the hidden logic that governs a situation. This is not about simply memorizing facts, but about discovering the underlying rules of a game or a system without being told what they are. Imagine walking into a room where objects must be sorted into bins, but no one tells you the sorting criteria. You have to guess, try a few things, see what works, and slowly deduce the rule. This process of inferring hidden structures is fundamental to how we navigate the world, from scientific reasoning to strategic decision-making. For machines to become truly adaptable, they need to master this same skill, moving beyond rigid programming to genuine conceptual understanding.

A team of researchers at Rutgers University and the University of Southern California set out to explore this challenge using a controlled environment called the Game of Hidden Rules. In this digital game, an artificial agent faces a board with nine objects, each having a specific shape and color, and four buckets in the corners. The goal is to move every object into the correct bucket, but the rule determining which object goes where is kept secret. The agent only learns through trial and error; if it makes a move, the game says "yes" or "no," offering no further explanation. The researchers wanted to see how well different types of artificial intelligence could figure out these invisible rules, how quickly they could learn them, and whether learning one rule would help them learn a new, related one later.

To test this, the team built two different ways for the computer to "see" the game board. The first method, called feature-centric, treats the board like a grid of lights. It looks at the board as a whole, noting where shapes and colors appear in specific spots, much like a camera taking a picture of the scene. The second method, called object-centric, treats each piece as a distinct individual. It focuses on the specific attributes of each object—its color, its shape, and its exact location—keeping them separate rather than blending them into a single image. The researchers trained both types of agents using a learning algorithm that rewards them for making correct moves and penalizes them for mistakes, allowing them to improve over thousands of attempts.

The results revealed a clear difference in how these two approaches handle learning. The object-centric agents, which viewed the world as a collection of individual items, proved to be much more effective at understanding the deeper logic of the game. They learned the rules faster and were better at applying what they learned to new situations. When the researchers tested how well the agents could generalize their knowledge, the object-centric models showed a remarkable ability to transfer their skills, though with limits. Statistical analysis showed that corresponding shape-based and color-based rules exhibited highly similar performance characteristics, indicating that the models treated these rule types consistently. However, when the board contained more objects than it had ever seen before, the object-centric models did experience a measurable degradation in performance, particularly for rules requiring complex ordering or relational reasoning. In contrast, the feature-centric agents struggled with these changes. They seemed to memorize the specific positions of objects on the board rather than understanding the abstract relationship between the objects and the buckets. When the board layout changed slightly, their performance dropped sharply, suggesting they had learned a map of the room rather than the rule of the game.

The study also looked at how difficult different types of rules were for the agents to grasp. Simple rules, like "put all red things in the first bucket," were learned quickly by both types of agents. However, rules that required understanding a sequence or a complex order, such as "place the pieces in the order they appear on a page," proved much harder. The researchers found that feature-ordering and conditional properties remained the most challenging for both models to learn, though the object-centric agents generally exhibited more stable and coherent learning curves across these difficult tasks compared to the feature-centric agents.

A particularly striking finding emerged when the researchers examined how the agents learned from one rule to the next. When an agent learned a simple rule and then was asked to learn a compound rule that combined two concepts, the object-centric agents showed a massive boost in performance, but only when the prior training specifically included the true component rules of the target. In these specific cases, they could reuse the knowledge they had gained, effectively skipping the initial trial-and-error phase with an almost five-fold improvement in convergence speed. The feature-centric agents, however, showed little to no benefit from this prior experience in most cases. They seemed unable to recognize that the new rule was built from parts they had already mastered, showing only weak positive transfer or modest gains in specific instances. This suggests that the object-centric approach allows the machine to build a more flexible and reusable understanding of concepts, similar to how a human might realize that the logic for sorting by one attribute is similar to the logic for sorting by another.

Beyond the machines, the researchers also looked at how humans played the game, specifically when they were given help from a computer program that offered suggestions. By analyzing the sequence of moves and the timing of successes and failures, they found that they could distinguish between a human playing alone and a human playing with assistance. Even without seeing the board or knowing the strategy, the pattern of wins and losses was enough to reveal the presence of the helper. This suggests that the way we learn and correct our mistakes leaves a unique signature, one that can be detected even when the underlying strategy is hidden.

Ultimately, this work highlights a crucial step forward in teaching machines to think conceptually. It suggests that for an artificial intelligence to truly learn from experience, it needs to see the world not just as a static image, but as a collection of interacting objects with their own properties. By focusing on the individual pieces and their relationships, machines can develop a more robust and transferable understanding of the rules that govern their environment. While the study does not claim to have solved the mystery of human-like learning, it provides strong evidence that the way we represent information to a machine is just as important as the learning algorithm itself. The object-centric approach appears to be a more promising path toward creating systems that can adapt, generalize, and learn from the hidden structures of the world around them.

Technical Summary: AI Learning and Conceptual Transfer in the Game of Hidden Rules

1. Problem Statement

This work addresses the challenge of latent rule learning within the Game of Hidden Rules (GOHR). In this environment, an agent must infer an unknown rule governing the assignment of objects to buckets based solely on sparse, binary feedback (accept/reject) after each action. The environment is partially observable; the agent does not see the rule directly but must deduce it through trial-and-error interaction.

The core research questions focus on:

How state representation (Feature-Centric vs. Object-Centric) affects learning performance, generalization, and transfer.
Which structural properties of rules (e.g., feature mapping, spatial ordering, conditional dependencies) contribute most to learning difficulty.
Whether conceptual transfer occurs between structurally related rules and how this transfer geometry is organized.
To what extent reinforcement learning (RL) agents exhibit behavioral patterns similar to human learners.

2. Methodology

2.1 Environment and Task

The GOHR environment consists of a $6 \times 6$ grid with four corner buckets and nine objects, each defined by shape and color. The agent selects an object and assigns it to a bucket. The hidden rule determines validity. Feedback is sparse: a move is either accepted (reward 0) or denied/immovable (reward -1). The episode ends when all objects are correctly placed or no moves remain.

2.2 State Representations

The study compares two distinct state encodings:

Feature-Centric (FC): The board is encoded as spatial feature maps ( $6 \times 6 \times 8$ channels) representing shape and color activations at grid positions. The model must infer associations between features and buckets through spatial alignment.
Object-Centric (OC): Each object is represented as an explicit vector containing one-hot encodings for color, shape, and position ( $x, y$ ). This representation binds attributes to specific objects, reducing the need for the model to infer object identity from spatial patterns.

Both representations incorporate temporal history, including the current state and the six most recent successful states, to aid in rule inference.

2.3 Architecture and Algorithm

The agents utilize a Transformer-based Actor-Critic (A2C) architecture:

Encoder: Processes the sequence of state histories using multi-head self-attention to capture dependencies among features, objects, and actions.
Heads: An Actor head outputs action probabilities (masked to valid moves), and a Critic head estimates the value function.
Training: The model is trained using Advantage Actor-Critic with entropy regularization and an $\epsilon$ -greedy exploration strategy. The discount factor is set very low ( $\gamma = 0.001$ ) to prioritize immediate error minimization over long-horizon rewards, reflecting the sparse, negative-only reward structure.

2.4 Evaluation Metrics

Performance is measured using three convergence metrics:

$M^\star$ : Move-based convergence (first move after which a fixed window of moves are all successful).
$E^\star_{mean}$ : Mean-window convergence (earliest episode where average error rate in a sliding window falls below a threshold).
$E^\star_{max}$ : Max-window convergence (stricter metric ensuring no errors in a sliding window).

3. Key Contributions

The paper makes the following contributions:

Representation Comparison: A systematic comparison of Object-Centric (OC) and Feature-Centric (FC) representations, demonstrating that OC generally yields more stable and interpretable learning behavior.
Rule Difficulty Analysis: A characterization of rule difficulty based on conceptual properties (e.g., feature mapping vs. ordering vs. conditionals), establishing that higher levels of abstraction and compositional complexity increase learning difficulty.
Transfer Learning Geometry: Extensive experiments on simple-to-simple and compound-rule transfer, revealing that transfer success depends heavily on structural alignment between predecessor and target rules.
Geometric Analysis: Application of hierarchical clustering, Multidimensional Scaling (MDS), and tanglegrams to visualize the "geometry of transfer," showing that OC forms coherent clusters of structurally related rules while FC exhibits more diffuse organization.
Human Learning Modeling: Development of a pseudo-bot-assisted human learning analysis using Markov/backoff models to classify gameplay sequences as assisted or unassisted based on temporal outcome patterns.
Software Implementation: A Gymnasium-compatible refactoring of the GOHR environment (though the paper notes that legacy history-augmented observations were required for successful learning).

4. Key Results

4.1 Learning Difficulty and Representation

FC Model: Learns positional properties (quadrant mapping, proximity) most easily but struggles with abstract feature ordering and conditional rules. It exhibits high sensitivity to training checkpoints and positional heuristics.
OC Model: Learns feature-based mappings and bucket orderings more effectively. While it also struggles with complex conditionals, the difficulty gaps between rule types are smaller than in FC. OC demonstrates more consistent learning curves across different rule families.
Generalization:
- FC: Shows poor generalization to out-of-distribution (OOD) board positions, suggesting it relies on memorized spatial templates rather than abstract rules.
- OC: Generalizes better to increased scene cardinality (16 objects vs. 9), though performance degrades for rules requiring complex relational reasoning over larger sets.

4.2 Transfer Learning

Simple-to-Simple Transfer: OC exhibits strong, semantically structured transfer within rule families (e.g., bucket-ordering rules transfer well to each other). FC transfer is unstable and often checkpoint-dependent.
Compound Transfer: When training on compound rules (e.g., combining reading-order and proximity), pretraining on the true component rules yields massive positive transfer (up to 5x faster convergence) for OC. FC shows weaker transfer benefits.
Negative Transfer: OC is highly sensitive to structural mismatches; pretraining on unrelated rules can cause severe negative transfer (slower convergence than training from scratch). FC is less sensitive to negative transfer but also less capable of positive transfer.

4.3 Transfer Geometry

OC: Clustering and MDS analyses reveal that OC organizes rules into coherent geometric clusters based on structural similarity. Predecessor influence and successor sensitivity are highly correlated (Adjusted Rand Index = 0.633), indicating a self-consistent transfer geometry.
FC: Exhibits weak alignment between predecessor and successor organization (Adjusted Rand Index = 0.168). The transfer geometry is diffuse, with rules like cw (clockwise bucket ordering) acting as outliers despite their conceptual similarity to other ordering rules.

4.4 Human Learning Analysis

A Markov/backoff model trained on outcome sequences (Accept/Deny) successfully distinguished between human-only and bot-assisted gameplay.
The model achieved up to 93.5% accuracy for fast-learning bot settings (ph4), demonstrating that temporal patterns of success and failure contain identifiable signatures of assistance, even without explicit board-state information.

5. Significance and Claims

The paper claims that explicit object-level representations (OC) are superior to spatial feature maps (FC) for learning hidden relational rules. The authors argue that OC facilitates the formation of reusable relational abstractions, enabling more systematic and coherent transfer learning.

Key takeaways include:

Structural Alignment is Critical: Successful transfer in hidden-rule environments depends primarily on the structural alignment between tasks, not just semantic similarity.
Representation Dictates Geometry: The choice of state representation fundamentally shapes the "geometry" of the learning space. OC creates a structured landscape where related concepts cluster, whereas FC results in a more chaotic, position-dependent landscape.
Human-AI Parallels: The analysis suggests that both human and AI learners exhibit similar sensitivities to rule complexity (ordering/conditionals are harder than feature mapping) and that simple probabilistic models can capture the behavioral signatures of learning assistance.

The authors maintain a modest tone, acknowledging that their work is exploratory and that the large number of statistical comparisons limits the ability to claim definitive confidence levels. They position this work as a step toward defining a "geometry of rules and concepts" for future principled exploration in relational reasoning and curriculum learning.