💬 NLP

It's About Time: Temporal References in Emergent Communication

This paper demonstrates that modifying the batching method in agent architecture, rather than altering the loss function, is sufficient to enable the emergence of temporal references in emergent communication systems, a capability essential for achieving more optimal coding efficiency.

Original authors: Olaf Lipinski, Adam J. Sobey, Federico Cerutti, Timothy J. Norman

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Olaf Lipinski, Adam J. Sobey, Federico Cerutti, Timothy J. Norman

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Teaching Robots to Say "Remember That?"

Imagine you are teaching two robots to talk to each other. They start with zero language, like two babies who have never heard a word. Their goal is simple: one robot (the Sender) sees an object, and the other robot (the Receiver) has to guess which object it is based on the message it receives.

Usually, in these experiments, the robots learn to describe the object's features. For example, if the object is a "red circle," the robot might invent a code that means "red" and another that means "circle."

The Missing Piece:
In human language, we do something very powerful: we talk about time. We say things like, "That's the same car we saw yesterday," or "Look at the bird from three seconds ago." This saves us a lot of effort. Instead of describing the car's color, speed, and model every single time, we just say, "The one from before."

The researchers wanted to know: Can robots invent this kind of "time-travel" language on their own? Can they learn to say, "That's the thing we saw earlier," without anyone explicitly teaching them to do so?

The Experiment: A Game of "Spot the Difference"

The researchers set up a game called a Temporal Referential Game.

The Setup: The Sender sees a sequence of objects. Sometimes, the object is brand new. Sometimes, it's an object the Sender saw a few turns ago.
The Goal: The Receiver has to pick the right object from a crowd of distractors.
The Challenge: If the object is a repeat, the Sender could just describe it again ("Red circle"). But if they invent a shortcut ("The one from two turns ago"), they save energy and time.

The Three Suspects: What Makes the Robots Talk About Time?

The researchers tested three different "ingredients" to see which one would make the robots start using time-based language:

The Environment (The "Repetition" Factor): They made the game repeat objects often.
- Analogy: Imagine playing a card game where the dealer keeps dealing the same Ace of Spades over and over. You'd naturally start saying, "It's that Ace again," instead of "It's an Ace of Spades."
- Result: Just repeating objects wasn't enough. The robots didn't automatically start using time shortcuts.
The Scorecard (The "Loss Function" Factor): They gave the robots a special reward (a bonus point) if they correctly identified when an object last appeared.
- Analogy: Imagine a teacher telling a student, "If you tell me what day you saw this apple, you get a gold star."
- Result: Even with the gold star, the robots didn't invent a time-language. They just got better at guessing the date, but didn't change how they spoke.
The Brain Structure (The "Architecture" Factor): This was the game-changer. They changed how the robots processed information.
- The Old Way (Base Agent): The robot looked at each object like a separate photo on a table. It didn't see the order.
- The New Way (Temporal Agent): The robot was forced to look at the objects like a movie reel. It had to watch them play out in a specific sequence, one after another, to understand the story.
- Analogy: The old robot was like someone looking at a pile of photos and trying to guess the story. The new robot was like someone watching a video; it naturally understood that "Scene 3" happens after "Scene 2."

The Discovery: It's All About How They "Think"

The paper found that only the robots with the "movie reel" brain structure (the architectural change) started inventing time-based language.

The Result: Over 95% of the robots with the new "sequential" brain developed a special code for "the thing we saw before."
The Surprise: They did this without needing extra rewards or special training. They just did it because their brain structure allowed them to see the flow of time.
The Efficiency: Once they invented this language, they used it frequently. It was like finding a shortcut in a maze. Instead of walking the whole path every time, they just pointed to the shortcut.

Why This Matters (According to the Paper)

The researchers argue that for robots to communicate as efficiently as humans, they need to be able to talk about the past.

Zipf's Law: In human language, the most common words are the shortest (e.g., "the," "is," "it").
The Robot Version: If a robot sees the same object often, it should invent a tiny, one-word code for "that one again." This saves "bandwidth" (like saving data on a phone).

The paper concludes that you can't just force robots to talk about time by changing the rules of the game or giving them bonuses. You have to build them with a brain that naturally understands sequences. If you give them a brain that sees time as a flow, they will naturally invent a language to talk about it.

Summary in a Nutshell

The Problem: Robots usually describe things from scratch, which is inefficient. They don't naturally say, "That's the one from before."
The Test: The researchers tried changing the game rules, giving bonuses, and changing the robot's brain.
The Winner: Only changing the brain structure (making them process information in a sequence, like a movie) made them invent time-based language.
The Lesson: To get robots to talk about time, don't just teach them the rules; build them to think in sequences.

Technical Summary: It's About Time: Temporal References in Emergent Communication

Problem Statement

Emergent communication research has extensively explored how autonomous agents develop bespoke languages to optimize efficiency, compositionality, and generalization. However, a critical gap exists: despite the known importance of temporal structure in natural human language, there is no existing evidence that agents in emergent communication settings develop temporal references—the ability to communicate about relationships between different time steps (e.g., referring to "the object seen two turns ago"). Without this capability, agents cannot coordinate on past events, potentially limiting bandwidth efficiency and task performance in scenarios requiring temporal understanding, such as turn-based strategy or multi-robot monitoring.

This paper addresses the question of when and how temporal references emerge in multi-agent systems. It specifically investigates whether this emergence is driven by environmental pressures (repetition of objects), external losses (explicit rewards for temporal prediction), or architectural biases (the agent's ability to process sequences).

Methodology

Experimental Environments

The authors utilize Referential Games (RG) as the foundational framework, where a sender observes a target object and sends a message to a receiver, who must identify the target among distractors. To introduce temporal dynamics, they propose Temporal Referential Games (TRG).

TRG Mechanism: In each episode, the sender observes either a new random object or a previously seen object from the history (within a defined horizon $h$ ).
Variants: The study includes standard RG, "Hard" variants (where targets differ from distractors by only one attribute), and specific TRG configurations with varying repetition probabilities ( $p$ ).
Control Environments: "Never Same" (no repetition) and "Always Same" (sequential repetition of a subset) are used to validate the emergence of temporal strategies.

Agent Architectures

Three distinct architectures are compared to isolate the factors driving temporal reference emergence:

Base Agent: A standard LSTM-based architecture (following the EGG framework) where objects are processed in parallel batches. Each object is treated independently, and the hidden state is reset or updated without explicit sequential ordering of the input sequence.
Temporal Agent: Introduces a sequentially batched LSTM. Instead of processing objects in parallel, the agent processes a sequence of objects (shape $[1, \text{batch\_size}, \text{features}]$ ) as a single temporal stream. This allows the network to build a representation of the sequence order.
TemporalR Agent: A hybrid combining the Base (parallel) and Temporal (sequential) approaches. It merges hidden states from both a regular LSTM and a sequentially batched LSTM via element-wise multiplication, allowing the agent to process object properties and their temporal order simultaneously.

Loss Functions and Training

Agents are trained with:

Standard Referential Loss: Cross-entropy between the receiver's guess and the sender's target.
Temporal Prediction Loss (Auxiliary): An additional loss term where the receiver predicts how many episodes ago an object was last seen. This tests if explicit external pressure aids emergence.

Evaluation Metrics

$M_{\ominus n}$ Metric: A novel metric designed to detect temporal references. It calculates the percentage of times a specific message is used exclusively when an object is a repetition of an object seen $n$ episodes ago. A score of 100% indicates a message is used strictly as a temporal operator (e.g., "previous object"), distinct from general object description.
Correctness: Standard task accuracy.
Compositionality: Measured via Topographic Similarity, Posdis, and Bosdis to ensure temporal referencing does not degrade the language's structural properties.

Key Results

Emergence of Temporal References

Architectural Necessity: The study finds that architectural change is the decisive factor. Agents using the Temporal and TemporalR architectures (sequential batching) develop temporal references in over 95% of runs, achieving 100% on the $M_{\ominus n}$ metric.
Insufficiency of Loss: Merely adding a temporal prediction loss to the Base architecture (parallel batching) does not lead to the emergence of temporal references (0% success rate). The ability to process inputs sequentially is a prerequisite.
Independence from Environment: Temporal references emerge in Temporal/TemporalR agents regardless of the training environment (RG or TRG) and repetition probability ( $p$ ). Even in standard RG environments without explicit temporal pressure, these agents develop temporal strategies.
Robustness: The emergence is robust across different repetition probabilities ( $p \in \{0.25, 0.5, 0.75\}$ ).

Task Performance and Compositionality

Accuracy: Contrary to the hypothesis that temporal references would improve task accuracy in temporal environments, the results show no significant accuracy gain for agents with temporal references compared to Base agents in TRG Hard environments. In some cases, agents trained with the auxiliary temporal loss performed slightly worse (approx. 1% drop), likely due to overfitting to temporal patterns at the expense of general object description.
Compositionality: The emergence of temporal references does not negatively impact the compositionality of the language. Agents maintain Topographic Similarity scores comparable to standard emergent languages. However, the authors note that standard compositionality metrics might slightly underestimate the quality of languages containing temporal references, as these references do not map directly to static object properties.

Significance and Claims

The paper claims to provide the first evidence of temporal references emerging in agent communication. Its primary contributions are:

Identification of the Mechanism: It establishes that architectural capacity (specifically, the ability to process data sequentially via sequential batching) is the necessary condition for temporal references to emerge. Neither environmental repetition nor external loss functions are sufficient on their own.
Efficiency Potential: The authors argue that temporal references offer a pathway to more efficient communication (closer to optimal coding) by allowing agents to use shorter, specialized messages for frequent events (repeated objects), analogous to Zipf's Law and Huffman coding.
Foundational Step: The work provides a controlled setting to investigate temporal communication strategies. It suggests that future improvements in emergent communication efficiency may require architectural designs that explicitly support temporal reasoning, rather than just optimizing loss functions.
Implications for Language Evolution: The findings draw parallels to human language evolution, suggesting that sequential learning mechanisms (like those in the Temporal agents) may be analogous to the cognitive processes required to encode and represent temporal sequences in natural language.

The paper concludes modestly, noting that while temporal references emerge naturally with the right architecture, their immediate impact on task accuracy in the specific environments tested was limited. However, the ability to reference past states is presented as a fundamental capability for future, more complex multi-agent coordination.