🤖 AI

Speaking Your Language: Spatial Relationships in Interpretable Emergent Communication

This paper demonstrates that agents can develop an interpretable emergent language capable of expressing spatial relationships with high accuracy, utilizing a mix of compositional and non-compositional messages that are successfully understood by both artificial receivers and humans.

Original authors: Olaf Lipinski, Adam J. Sobey, Federico Cerutti, Timothy J. Norman

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Olaf Lipinski, Adam J. Sobey, Federico Cerutti, Timothy J. Norman

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are playing a game of "Guess the Object" with a friend, but you can’t see the same things. You are looking at a long line of numbered tiles, but you can only see a tiny window of five tiles at a time. Your friend sees the whole line and a few other random numbers. Your job is to send a short, secret code to your friend so they can pick out the exact tile you are looking at.

This paper is about teaching computer programs (called "agents") to invent their own secret language to play this game. But instead of just memorizing codes for every single tile, the researchers wanted to see if the computers would learn to use spatial relationships—basically, learning to say things like "the number to the left of 5" or "the one at the very beginning."

Here is how the story unfolds, explained with some everyday analogies:

1. The Setup: The "Window" Problem

Think of the sequence of numbers as a long train. The "Sender" agent is standing on a platform with a small window that only lets them see five train cars at once. One of those cars has a special mark (a -1) indicating the target. The "Receiver" agent sees the entire train and a list of four possible cars (one correct, three wrong).

The Sender doesn’t know where the train starts or ends, only what’s in their little window. To tell the Receiver which car is the target, the Sender can’t just say "It’s car #12" because they don’t know the global number. They have to describe it relative to what they see, like "It’s the car right after the one with the number 4."

2. The Discovery: Two Types of "Words"

The researchers let the agents play this game thousands of times. They didn’t teach them English; the agents had to invent their own symbols (numbers from 1 to 26). After training, the agents became incredibly good at the game, getting over 90% of the answers right.

But the real question was: What does their language look like? To find out, the researchers used a statistical tool called NPMI (think of it as a "word association detector") to see which symbols were linked to which situations. They found that the agents developed two distinct styles of communication:

Style A: The "Shorthand" (Non-Compositional)

Sometimes, the agents used a single, unique code for a very specific situation.

Analogy: Imagine if you had a special hand signal that meant "I am at the very start of the line." You wouldn’t say "I am at the start"; you’d just do the signal.
In the paper: The agents created specific codes for rare events, like when the target was at the very beginning or very end of the sequence. Because these situations are rare, it’s more efficient to have one special "word" for them rather than building a complex sentence.

Style B: The "Lego Blocks" (Compositional)

Most of the time, the agents used a mix-and-match approach. They combined different symbols to build a meaning.

Analogy: Think of building a sentence with Lego bricks. One brick means "2 steps to the right," and another brick means "the number 18." If you snap them together in the right order, you get the message: "The target is 2 steps to the right of 18."
In the paper: The agents learned that the position of the symbol in the message mattered. If the symbol for "2 steps right" was in the first slot, it meant one thing; if it was in the second slot, it might mean something else. This is a primitive form of syntax (grammar), where word order changes meaning.

3. The Proof: Can Humans Understand It?

The researchers didn’t just guess what the agents meant. They tested their theory by creating a "dictionary" based on their statistical analysis. Then, they used this dictionary to send messages to the Receiver agent.

The Test: They took the "Lego blocks" the agents had invented, assembled them into new messages using the rules they discovered, and sent them to the Receiver.
The Result: The Receiver understood these human-assembled messages correctly over 78% of the time. This proves that the agents weren’t just using random noise; they had developed a structured, interpretable language that humans could decode.

4. Why Does This Matter?

The paper argues that this is a big deal because previous AI communication experiments often resulted in languages that were efficient but completely unintelligible to humans—like a secret code that even the creators couldn’t crack.

By forcing the agents to deal with spatial relationships (left, right, beginning, end), the researchers coaxed the AI into developing a language that mirrors a fundamental part of human speech: deixis (pointing to things in space or time). It shows that even in a simple game, AI can learn to use grammar-like structures and spatial references, making their communication more transparent and trustworthy.

In Summary

The paper shows that when AI agents are forced to describe where things are relative to other things, they naturally invent a language that uses both special shorthand codes for rare events and mix-and-match grammatical structures for common ones. And best of all, we can actually read their dictionary and understand what they’re saying.

Technical Summary: Speaking Your Language: Spatial Relationships in Interpretable Emergent Communication

Problem Statement
Emergent communication research has demonstrated the development of efficient and compositional languages among artificial agents. However, these emergent languages often lack fundamental properties found in human language, such as syntax, recursion, and deixis (pointing through language). Specifically, while temporal deixis has been investigated, no prior work has demonstrated the emergence of spatial deixis—the ability to refer to specific locations within an observation relative to others. Spatial referencing is critical for establishing shared context, improving communication efficiency by reducing descriptive redundancy, and enhancing adaptability in dynamic environments. This paper addresses the gap by investigating whether agents can develop a language capable of expressing spatial relationships within their observations.

Methodology
The authors introduce a modified referential game environment designed to necessitate spatial communication.

Environment: The sender observes a window of length 5 from a longer sequence of unique integers ( $0 \dots 59$ ). One integer in the window is marked as the target with $-1$. The sender does not know the global position of the target within the full sequence, which is visible only to the receiver. Consequently, the sender must communicate the target’s position relative to other integers in the window (e.g., "the integer to the right of 3"). The receiver observes the full sequence, the sender’s message, and a set of distractors, and must identify the target.
Agent Architecture: The agents utilize a standard Emergent Communication Game (EGG) architecture with GRUs. The sender encodes the observation window into a 3-integer message ( $m \in \{1 \dots 26\}^3$ ). The receiver processes the message and the full sequence to predict the target’s index among the distractors.
Analysis Framework: To interpret the discrete emergent language, the authors employ Normalised Pointwise Mutual Information (NPMI), a collocation measure from computational linguistics. They propose two algorithms:
1. PMInc: Measures non-compositional (monolithic) messages, where the entire message corresponds to a specific contextual feature (e.g., target at the beginning of the sequence).
2. PMIc: Measures trivially compositional messages by analyzing n-grams (unigrams and bigrams) to identify separate components for integer references and positional references.

Key Contributions

Demonstration of Spatial Deixis: The paper provides the first evidence that agents in a referential game can develop spatial references, moving beyond simple object identification to relational description.
Interpretability Method: The authors introduce an NPMI-based segmentation and interpretation method that maps emergent message segments to specific meanings (integers and relative positions) without requiring pre-defined labels for spatial relationships.
Identification of Mixed Compositionality: The analysis reveals that agents use a mixture of non-compositional and compositional strategies. Non-compositional messages are used for sparse, specific positional contexts (e.g., sequence boundaries), while compositional messages are used for general spatial relationships.

Results

Communication Accuracy: Agent pairs achieved over 90% accuracy in the referential task, confirming the successful development of a functional spatial language.
Emergence Patterns:
- Non-Compositional Positional References: In 99.3% of runs, agents developed unique messages for specific positional contexts (begin, begin+1, end-1, end). These messages were often monolithic and reserved exclusively for these contexts.
- Compositional References: In 25% of runs, agents developed compositional messages consisting of a positional component (e.g., "2 right of") and an integer component (e.g., "integer 18").
Interpretability Validation: The authors tested the validity of their NPMI-derived dictionary by querying the receiver agent with messages constructed from the interpreted lexicon.
- For non-compositional positional messages, the receiver achieved an average accuracy of 90% (max 94%), significantly above chance (20%).
- For compositional messages, including the identified positional components increased accuracy from near-chance levels (22%) to 30% on average, with some agent pairs exceeding 78%. This confirms that the NPMI method correctly identified the semantic contributions of positional n-grams.

Significance and Claims
The paper claims that the emergence of compositional spatial references represents a "first hint of a syntactic structure" in emergent language, as specific n-grams were assigned fixed positions within the message (e.g., positional reference always first, integer reference always second). The authors argue that the coexistence of non-compositional and compositional strategies is driven by efficiency: non-compositional messages are more compressible and thus advantageous for sparse observations (like sequence boundaries), whereas compositional messages are necessary for generalizing across the majority of observations. The work suggests that careful design of simple referential games can elicit fundamental linguistic properties, such as syntax and spatial deixis, contributing to the development of trustworthy and human-interpretable emergent communication systems.