💻 computer science

Nemobot Games: Crafting Strategic AI Gaming Agents for Interactive Learning with Large Language Models

This paper introduces Nemobot, an interactive agentic environment that leverages large language models to extend Shannon's taxonomy of game-playing machines, enabling the creation and self-programming of strategic AI agents across dictionary-based, solvable, heuristic, and learning-based games through methods like mathematical reasoning, crowd-sourced data synthesis, and reinforcement learning with human feedback.

Original authors: Chee Wei Tan, Yuchen Wang, Shangxin Guo

Published 2026-04-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chee Wei Tan, Yuchen Wang, Shangxin Guo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you want to teach a computer how to play a game, but instead of writing thousands of lines of rigid code like "If X happens, do Y," you want to teach it by talking to it, letting it learn from mistakes, and even letting it write its own rules as it goes.

That is the core idea behind Nemobot, a new tool described in this paper. Think of Nemobot not just as a game bot, but as a digital playground and a classroom combined, where humans and Artificial Intelligence (AI) work together to build smarter game players.

Here is a breakdown of how it works, using simple analogies:

1. The Old Way vs. The New Way

The Old Way (The Encyclopedia):
In the past, to make a computer play a simple game like Tic-Tac-Toe, programmers had to create a massive "dictionary" or encyclopedia. They would write down every single possible move the computer could ever make. It was like giving the computer a giant book of answers. If the game got too complex (like Chess), the book would be too huge to carry.

The New Way (The Smart Tutor):
Nemobot uses Large Language Models (LLMs)—the same technology behind chatbots like me. Instead of a giant book of answers, the AI has a "brain" that understands the rules and the logic of the game.

Analogy: Imagine teaching a child to play chess. Instead of memorizing every possible game scenario, you teach them the rules and how to think strategically. If they make a mistake, you explain why it was a mistake. Nemobot does this, but the "child" is the AI, and the "teacher" is the human programmer using natural language.

2. The Four Levels of the Game (Shannon's Taxonomy)

The paper updates a famous 70-year-old idea by Claude Shannon, who classified game-playing machines into four types. Nemobot brings all four to life using modern AI:

Level 1: The Dictionary (Tic-Tac-Toe)
- The Concept: The AI knows every possible move.
- The Nemobot Twist: Instead of storing a massive database, the AI "remembers" the patterns. It's like a chef who doesn't need a recipe book for a simple salad because they know exactly how to mix the ingredients. The AI compresses all that knowledge into its brain, saving space and making it faster.
Level 2: The Math Wizard (Nim)
- The Concept: Some games are purely mathematical. There is one perfect way to win.
- The Nemobot Twist: The AI doesn't just calculate the answer; it acts like a tutor. It solves the math problem and then explains it to you in plain English: "I took 3 stones because that leaves you with a number that guarantees I win next turn." It turns cold math into a conversation.
Level 3: The Strategist (Mancala)
- The Concept: These games are too complex for pure math. You need to guess what the opponent will do.
- The Nemobot Twist: The AI combines classic computer algorithms (like looking ahead in a game tree) with crowdsourcing. It learns from thousands of real human players.
- Analogy: Imagine a coach who watches thousands of games, collects the best moves from the crowd, and then tells the AI, "Hey, when the board looks like this, humans usually do this." The AI blends the computer's speed with human creativity.
Level 4: The Learner (Self-Improving)
- The Concept: The AI learns by trial and error.
- The Nemobot Twist: The AI plays against itself or humans, makes mistakes, gets "critiqued," and tries again. It's like a video game character that levels up every time it dies. The AI uses reinforcement learning (getting a reward for winning, a penalty for losing) to refine its own strategy over time.

3. The "Crowdsourced" Superpower

One of the coolest parts of Nemobot is how it uses human creativity.

The Analogy: Think of it like a Wikipedia for game strategies. Instead of one genius programmer writing the code, thousands of students and players contribute their ideas. They write prompts (instructions) for the AI.
If a player figures out a clever trick in a game, they can teach the AI how to use it. The AI then "reads" these tips and updates its own logic. This creates a cycle where the more people play, the smarter the AI gets.

4. Why Does This Matter? (The "Self-Programming" Dream)

The paper's ultimate goal is Self-Programming AI.

The Vision: In the 1950s, a pioneer named Claude Shannon asked, "Can we build a computer that writes its own instructions?"
The Nemobot Reality: Nemobot is a step toward that dream. It allows the AI to not just play the game, but to rewrite its own strategy guide based on what it learns.
The Metaphor: Imagine a student who, after failing a test, doesn't just ask the teacher for the right answer, but rewrites their own textbook to ensure they never make that mistake again. That is what Nemobot enables.

5. Who Is This For?

This isn't just for computer scientists.

For Students: It's a learning tool. You can learn how AI works by playing with it, tweaking its "brain" (the prompts), and seeing what happens.
For Developers: It's a sandbox. You can build your own games and AI agents without needing to be a coding wizard, because the AI helps you write the code.

Summary

Nemobot is a bridge between the rigid, old-school way of programming computers and the new, flexible world of AI. It turns game-playing into a collaborative conversation between humans and machines, where the AI learns from us, we learn from the AI, and together, we create strategies that are smarter than either of us could build alone. It's not just about winning the game; it's about teaching the computer how to learn how to win.

1. Problem Statement

The paper addresses a critical gap in the intersection of Artificial Intelligence and game programming: the lack of a structured, reproducible, and educational framework for leveraging Large Language Models (LLMs) in strategic game development.

The Limitation of Current Approaches: While LLMs can play games when prompted directly, their behavior is often non-deterministic, opaque, and difficult to debug or systematically improve. Existing AI game agents (like AlphaGo) are often "black boxes" that do not allow for human inspection or modification of their internal logic.
The Theoretical Gap: Claude Shannon's seminal taxonomy of game-playing machines (Dictionary, Formula-based, Heuristic, and Learning machines) was developed in an era of deterministic algorithms. Modern LLM-based systems are inherently probabilistic. There is no established paradigm for mapping Shannon's classical categories onto modern LLM capabilities in a way that supports self-programming AI and interactive learning.
Educational Barrier: Traditional AI game programming requires deep expertise in algorithms (e.g., Minimax, Reinforcement Learning) and code implementation, creating a high barrier to entry for students and novice developers.

2. Methodology

The authors propose Nemobot, an interactive agentic engineering environment that operationalizes Shannon's taxonomy using LLMs as modular, programmable components.

A. Theoretical Framework: Extending Shannon's Taxonomy

The paper reinterprets Shannon's four types of game-playing machines through the lens of LLM capabilities:

Dictionary-type Machines: Instead of storing exhaustive state-action pairs in memory, the LLM acts as a compressed, generalized model. It infers optimal moves for unseen states based on pre-trained knowledge, effectively "neuralizing" the dictionary.
Formula-based Machines (Rigorous Math): For solvable games (e.g., Nim), the LLM acts as an interactive copilot. It does not just compute the solution but generates code for mathematical algorithms (e.g., GCD for Euclid's game) and provides natural language explanations of the reasoning behind optimal moves.
Heuristic-based Machines: For complex games (e.g., Mancala), the LLM synthesizes strategies by combining classical algorithms (like Minimax) with crowdsourced data. It uses Chain-of-Thought reasoning to break down complex decision spaces and retrieve relevant strategies from a game tree.
Learning Machines: The LLM facilitates Reinforcement Learning (RL) with Human Feedback (RLHF). It acts as an AI coach, analyzing trial-and-error outcomes, critiquing moves, and refining strategies iteratively.

B. System Architecture: Nemobot

Nemobot is a web-based platform (deployable on Messenger/Telegram) designed with three core layers:

Model-Agnostic Integration: Uses a "Bring Your Own Key" (BYOK) approach, allowing users to connect various LLMs (GPT-4, GPT-3.5, etc.) via API.
State Management: Game logic and state management run locally (NodeJS) to ensure low latency. The LLM is queried only when specific "LLM functions" are triggered for strategy or reasoning, preventing inference latency from bottlenecking the game loop.
Data Persistence & Crowdsourcing: Game outcomes and performance metrics are asynchronously synced to a cloud database, enabling the aggregation of crowdsourced training data across users.

C. Workflow: Collaborative Prompt Engineering

The core innovation is Programmable Prompt Engineering:

Modular LLM Functions: Users define LLM logic (heuristics) using natural language or pseudo-code rather than hard-coded rules.
Iterative Training Loop (Algorithm 1):
1. Define an initial heuristic ( $H_0$ ).
2. Execute games (via Chat Playground or automated simulation).
3. Collect rewards ( $R$ ) and analyze performance.
4. Update the heuristic ( $H_k$ ) based on feedback (either manual refinement or automated RL updates).
5. Repeat until performance thresholds are met.
Crowdsourcing: The system aggregates data from human players and diverse AI strategies to refine prompts and heuristics, creating a self-reinforcing cycle of improvement.

3. Key Contributions

Nemobot Framework: A novel programming environment that treats LLM functions as inspectable, modular subroutines, bridging the gap between abstract AI theory and practical game development.
Taxonomy Extension: A systematic mapping of Shannon's four machine types to LLM capabilities, demonstrating how LLMs can compress dictionaries, explain mathematical proofs, synthesize heuristics, and drive reinforcement learning.
Collaborative Prompt Engineering: A methodology for using crowdsourcing to refine AI strategies. This transforms prompt engineering from a static task into a dynamic, community-driven optimization process.
Educational Impact: A tiered curriculum (from dictionary-based to learning-based agents) that allows students to learn AI concepts (heuristic design, RL, code generation) through hands-on game programming.

4. Results and Evaluation

The framework was evaluated in undergraduate courses at City University of Hong Kong, Nanyang Technological University, and a remote internship program with Princeton University (totaling ~360 students).

Implementation Success: Students successfully programmed diverse games including:
- Tic-Tac-Toe: Demonstrated LLMs acting as a compressed dictionary, generating optimal moves without exhaustive state storage.
- Nim: Showcased LLMs generating code for mathematical strategies (Nim-sum) and explaining the logic in natural language.
- Mancala: Illustrated the use of LLMs to query game trees and integrate crowdsourced data for heuristic refinement.
- Advanced Applications: Included physics quizzes, RPGs with dynamic dialogue, and audio tagging systems.
Performance Metrics:
- The system successfully reduced memory overhead for dictionary-based games by replacing explicit storage with LLM inference.
- In the Nim game simulation, the system utilized Michie's Boxes algorithm (a precursor to modern RL) combined with LLM fine-tuning to master strategies through trial-and-error.
- A Mancala Leaderboard was established where human players competed against AI agents using different tree depths, providing rich data for the LLM to refine its decision-making.
Educational Outcomes: The platform enabled students with varying expertise levels to design, test, and deploy AI agents, fostering a deeper understanding of strategic reasoning and AI mechanics.

5. Significance and Future Directions

Paradigm Shift: Nemobot moves the field from "prompting an LLM to play" to "programming an LLM to learn and evolve." It operationalizes the vision of self-programming AI, where agents iteratively refine their own logic based on interaction and feedback.
Educational Utility: It serves as a powerful tool for gamified AI education, making complex concepts like Reinforcement Learning and Heuristic Search accessible through natural language interfaces.
Broader Applications: The framework extends beyond gaming to knowledge tracing and adaptive learning systems. The "trial-and-error" mechanism can model student mastery, while crowdsourcing can help meta-learn the best pedagogical strategies.
Future Challenges: The authors identify key areas for future work, including:
- Improving long-term strategic planning in complex games (e.g., Chess) via hierarchical systems.
- Balancing generalization vs. specialization in fine-tuning.
- Addressing scalability issues in human feedback loops.
- Ensuring ethical design to prevent adversarial exploitation of human weaknesses.

In summary, the paper presents Nemobot as a transformative step toward Agentic Engineering, demonstrating how LLMs can be structured into reproducible, educational, and self-improving systems that honor the foundational theories of AI while leveraging modern generative capabilities.