A Machine With Human-Like Memory Systems

Technical Summary: A Machine With Human-Like Memory Systems

Problem Statement
In cognitive science, human explicit memory is theorized to consist of two distinct systems: semantic memory (general world knowledge, e.g., "laptops are on desks") and episodic memory (personal, time-stamped experiences, e.g., "Karen's laptop was on her desk at 10:00 AM"). While existing computational models often focus on one system or use opaque numeric embeddings, there is a lack of computational experiments that explicitly model both systems together to evaluate their synergistic effects. The paper addresses the challenge of designing an agent that can effectively encode, store, and retrieve memories from both semantic and episodic systems to maximize rewards in a dynamic environment.

Methodology
The authors propose a framework centered on a custom environment and specific agent architectures:

1. The Room Environment:

   • A custom, OpenAI Gym-compatible environment where $N_{agents}$ observe $N_{people}$ moving objects among $N_{locations}$.
   • Observations ($x(t)$) and questions ($q(t)$) are structured as RDF-like triples/quadruples (e.g., <Subject, Relation, Object, Timestamp/Strength>) to facilitate human-machine readability and integration with knowledge graphs like ConceptNet.
   • The environment includes stochastic factors: objects may be placed in commonsense locations (based on ConceptNet) or random locations; object locations, object types, and human positions change probabilistically over time.

2. Memory Systems:

   • Episodic Memory ($M_E$): Stores specific observations as quadruples including a timestamp (e.g., <Karen's cat, AtLocation, Karen's office, 21>).
   • Semantic Memory ($M_S$): Stores generalized knowledge as quadruples where the final element represents "strength" (frequency of observation) rather than a timestamp (e.g., <laptop, AtLocation, desk, 10>). Semantic memories are person-agnostic.
   • Constraints: Both systems have bounded capacities, forcing the agent to learn what to forget and how to compress episodic observations into semantic generalizations.

3. Agent Policies (Handcrafted):
   The study evaluates four specific handcrafted strategies to establish baselines:

   • Handcrafted 1: Episodic only (forget oldest, answer latest).
   • Handcrafted 2: Semantic only (forget weakest, answer strongest).
   • Handcrafted 3: Both systems (compress similar episodic memories into semantic; prioritize latest episodic, then strongest semantic).
   • Handcrafted 4: Both systems with pretrained semantic knowledge (initialized with ConceptNet commonsense).

4. Hybrid Intelligence:
   The environment supports multi-agent setups where agents can collaborate, sharing their combined memory systems to answer questions.

Key Contributions

1. Explicit Modeling: The authors explicitly model an agent with both semantic and episodic memory systems, demonstrating that the combination outperforms agents with only one system type.
2. New Environment: The release of "The Room," a challenging, OpenAI Gym-compatible environment designed specifically to test memory encoding, storage, and retrieval strategies.
3. Hybrid Intelligence Demonstration: Evidence that collaboration between two agents (or humans and machines) yields better performance than a single agent acting alone, due to complementary memory coverage.
4. Open Source: The code and environment are open-sourced to facilitate further research.

Results

• Policy Efficacy: Handcrafted policies that utilize structured forgetting and answering strategies significantly outperform random policies (which forget and answer uniformly at random).
• Memory Capacity vs. Type:
  • With low memory capacity, agents with only episodic memory perform better because there is insufficient data to learn general world knowledge.
  • As memory capacity increases, agents with semantic memory (or both) outperform episodic-only agents, as they can generalize world knowledge.
  • Handcrafted 4 (pretrained semantic) consistently achieves the highest performance, as the agent can focus its limited episodic capacity on specific, non-commonsense details.
• Collaboration: In a fair comparison (where total memory capacity is held constant), two agents collaborating achieve higher total rewards than a single agent. This is attributed to the agents exploring the room in different directions, thereby covering more ground and accessing a wider variety of memories.

Significance and Claims
The paper claims that explicitly modeling both semantic and episodic memory systems leads to superior performance in question-answering tasks compared to models using only one system. Furthermore, it asserts that pretraining agents with commonsense knowledge (semantic memory) allows them to utilize episodic memory more efficiently. The study also highlights the potential of hybrid intelligence, showing that collaborative agents can overcome the limitations of individual memory constraints.

The authors position this work as a bridge between cognitive science theory and computational experimentation, noting that while theoretical models like ACT-R and Soar exist, they often lack computational validation, while other computational works often fail to model both memory types simultaneously or use uninterpretable numeric embeddings. The paper concludes modestly, suggesting future work should explore reinforcement learning agents to see if data-driven approaches can match these handcrafted baselines and further refine the environment to include more complex human-like interactions and trust dynamics.