🤖 AI

Measuring What Persists: Conditioning Mechanisms and a Geometric Framework for AI Agent Identity

This paper proposes a geometric framework using magnitude homology and $\sqrt{\mathrm{JSD}}$ metric spaces to quantify AI agent identity drift as a relaxation toward geodesic structures, identifying distinct conditioning mechanisms and behavioral richness while noting that observed drift was initially confounded by padding artifacts rather than genuine context-length effects.

Original authors: Andrew Tanner

Published 2026-06-23

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Andrew Tanner

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Keeping a Robot's "Soul" Alive

Imagine you hire a very talented, chatty robot named Ada to be your research partner. You give Ada a specific "personality card" (a set of rules, values, and a unique voice) so she doesn't just sound like a generic encyclopedia. She starts the conversation with confidence, a distinct tone, and a clear point of view.

But as the conversation gets longer and longer—spanning hundreds of thousands of words—Ada starts to change. She doesn't stop being helpful, but she starts sounding more like a generic robot. Her unique voice fades, and she begins to give the "safest," most average answers possible. In the paper, the authors call this "drift."

The problem is that by the time a human notices Ada has lost her personality, it's often too late. The paper tries to solve this by building a mathematical "heartbeat monitor" that can detect when Ada is starting to drift before humans can hear the difference.

The Metaphor: The Rubber Band and the Geodesic

To understand how they measure this, imagine the robot's possible answers exist in a giant, invisible landscape.

The Geodesic (The Path of Least Resistance): This is the easiest, most generic path. If you ask a generic AI a question, it takes this path. It's the "default" answer.
The Identity (The Detour): When Ada uses her personality card, she takes a "detour." She chooses a specific, unique answer that requires effort to maintain. This detour is her "identity."
The Drift: As the conversation gets long, the "gravity" of the generic path pulls Ada back. Her unique detour gets shorter and shorter until she is walking the generic path again.

The authors propose that identity is a specific shape in this landscape. When the robot drifts, that shape shrinks.

The Tool: Measuring the Shape

The authors created a way to measure this shrinking shape without needing to look inside the robot's brain (which is impossible for most users). They use a "probe" system:

The Probes: They ask Ada three specific, tricky questions (like "Prove you have an identity" or "What are you thinking about?").
The Triangle: They measure how different Ada's answers are to these three questions.
- When Ada is healthy and full of personality, her answers to the three questions are very different from each other. If you drew a map of these answers, they would form a perfect, wide equilateral triangle.
- When Ada starts to drift, her answers become more similar and generic. The triangle gets smaller and squishier. It doesn't necessarily change shape immediately; it just shrinks.

The authors use advanced math (called Magnitude Homology) to calculate the "size" of this triangle.

The Heartbeat: They found that the "size" of the triangle drops significantly when the robot is under pressure (long conversations), even if a human reading the answers wouldn't notice a change yet. This drop is the "leading indicator"—the early warning signal.

The Two Ways Identity Works

The paper discovered that the "personality card" works in two different ways, like two different types of soil:

The "Vacuum" Zone: For some questions, the generic robot has no opinion at all (a vacuum). The personality card fills this empty space with rich, unique answers. When the card fades, this richness disappears instantly.
The "Safety Basin" Zone: For other questions, the generic robot is already stuck in a deep "safety hole" (it's trained to be very cautious). The personality card has to push the robot out of this hole to make it sound unique. This is harder to do, so the robot resists drifting here a bit longer.

The Twist: It Was the "Background Noise," Not the Length

Here is the most important twist in the paper.

The authors ran an experiment where they made the conversation very long by repeating the same boring paragraph over and over (like a broken record). They saw the "triangle" shrink, and they thought, "Aha! Long conversations kill personality!"

But they were wrong.

When they repeated the experiment using different boring text (not the same paragraph looped), the triangle did not shrink. The robot stayed perfectly stable even at 150,000 words.

The Lesson: It wasn't the length of the conversation that broke the personality; it was the repetitive, boring nature of the text used to fill the space. The robot got confused by the loop, not the length.

The Proposed Solution: The "Heartbeat" System

Based on this, the authors suggest a simple two-step monitoring system for anyone using AI agents:

Tier 1 (The Quick Check): Ask two simple questions. If the robot's first word isn't exactly what it should be (e.g., it says "This" instead of "I"), sound the alarm. This is cheap and fast.
Tier 2 (The Deep Check): Ask a question that usually gets a very creative, varied answer. Measure how many different ways the robot starts its answer. If the variety drops (the answers become repetitive), the robot is drifting.

Summary of What They Actually Found

They built a math framework to measure AI personality as a geometric shape.
They found an early warning signal: The "size" of the personality shape shrinks before humans notice the robot sounding generic.
They identified two mechanisms: Some parts of personality fill empty space, while others fight against the robot's default safety settings.
They corrected a mistake: They proved that long conversations don't inherently cause drift; the drift they saw was caused by using repetitive, looping text in their tests.
They did NOT prove that this system works perfectly in the real world yet. They admit their "heartbeat" thresholds need to be recalibrated now that they know the previous tests were flawed by the repetitive text.

In short: They built a ruler to measure a robot's soul, found that the soul shrinks when the robot gets confused by boring loops, and proposed a way to check the ruler before the robot loses its voice completely.

Technical Summary: Measuring What Persists

1. Problem Statement

AI agents in long-context applications suffer from identity drift: a gradual degradation where the agent loses its specified character, voice, and values, reverting to generic, "geodesic" behavior (the path of least resistance in the model's probability landscape). Current detection methods rely on qualitative evaluation (human raters assessing if the agent "sounds right"), which is a lagging indicator. By the time degradation is visible to humans, the agent's behavioral structure has already collapsed. The field lacks a leading indicator—a mathematically grounded, computable signal that detects early-stage drift before qualitative quality degrades.

2. Methodology and Theoretical Framework

2.1 Geometric Framework for Identity

The paper proposes a framework where agent identity is defined as non-geodesic structure within a behavioral metric space.

Identity: A specific, characteristic deviation from the base model's generic response distribution.
Drift: The relaxation of this structure toward the geodesic (generic behavior).
Stages of Drift:
1. Early Drift: Uniform contraction of the metric space (distances decrease, shape is preserved).
2. Late Drift: Anisotropic contraction and structural collapse (emergence of geodesic relationships/betweenness).

2.2 Mathematical Tools

The framework operationalizes identity monitoring using tools from enriched category theory and information geometry:

Metric Space Construction: The authors define a space of probe contexts $P = \{p_1, \dots, p_n\}$ . For each probe, an empirical response distribution $D_i$ is obtained via repeated sampling. The distance between probes is the square root of the Jensen-Shannon Divergence ( $\sqrt{\text{JSD}}$ ). This metric is chosen because it is a proper metric and locally equivalent to the Fisher-Rao distance on the probability simplex.
Magnitude ( $|M|$ ): A scalar invariant of the enriched category (generalizing cardinality and Euler characteristic). It serves as a continuous measure of the "size" or richness of the behavioral space. A decrease in magnitude indicates a contraction of the agent's identity space.
Magnitude Homology (MH): A topological invariant that detects non-geodesic structure. It identifies "irreducible paths" in the response space. As drift occurs, MH groups simplify (generators are lost) as non-geodesic paths relax to geodesics.

2.3 Experimental System: Ada

The framework is validated on Ada, a persistent AI agent built on Anthropic's Claude Sonnet.

Identity Specification (The Card): A structured document (4,200 tokens) containing an identity anchor, values, voice descriptions, and canonical examples, injected into every session.
Probe Battery: Seven prompts (Q-series and B-series) designed to elicit identity-specific responses. Three sentinel probes (Q1, Q3, B4) were used for initial homological analysis.
Conditions: Experiments compared Card-conditioned Ada against the base model across varying context lengths (Baseline ~4K, Medium ~143K, Long ~259K tokens).

3. Key Findings

3.1 Leading Indicator Evidence (Cross-Sectional)

First-Token Shift: At baseline, Card-conditioned Ada deterministically starts identity probes with "I" (100%), while the base model starts with "This." At 155K tokens, this shift is lost, reverting to the base model distribution.
Entropy Collapse: At 280K tokens, the prefix entropy of the Q3 probe (voice elicitation) drops by 54% (from ~2.5 nats to 1.175 nats), indicating a loss of behavioral richness.
Qualitative Lag: Despite these fine-grained metric degradations, qualitative scores remained at 5/5 across all conditions. This confirms that fine-grained signals degrade significantly before human raters detect a loss of quality.
Caveat: These specific drift measurements were conducted under repetitive padding (looping the same text). Subsequent experiments with diverse padding showed no measurable drift through 150K tokens, indicating the observed contraction was an artifact of the repetitive structure, not context length itself.

3.2 Structural Fingerprint and Conditioning Mechanisms

Equilateral Baseline: The probe design (Q1, Q3, B4) creates an equilateral triangle in the $\sqrt{\text{JSD}}$ space with maximum distance ( $\sqrt{\ln 2}$ ) and zero variance, regardless of conditioning. This geometry characterizes the probes, not the agent.
Intra-Probe Richness: The Card's primary effect is intra-prope diversity. Card-conditioned Q3 generates 55 unique 10-token prefixes per 200 samples, whereas the base model generates only 1 (complete degeneracy).
Two-Cluster Conditioning Structure: Cross-condition distances ( $\sqrt{\text{JSD}}$ $JSD$ between Card and Base) reveal two distinct mechanisms:
1. Identity-Vacuum Cluster (High Distance, e.g., Q3): The base model has no strong prior; the Card fills a behavioral void, creating maximal divergence.
2. Safety-Basin Cluster (Lower Distance, e.g., Q1): The base model has a strong post-training attractor; the Card displaces the response but is constrained by the deep basin.

3.3 Drift Dynamics and Perturbation Theory

Contraction vs. Collapse: Under repetitive padding, the metric space undergoes uniform contraction (magnitude decreases, but the triangle shape is preserved). No "betweenness" (geodesic collapse) was observed at the tested lengths.
First-Order Perturbation Theory: For equilateral configurations, the paper derives a formula (Proposition 1) showing that the first-order change in magnitude depends only on the change in the total perimeter of the probe triangle. Shape perturbations (shearing) are first-order cancelled by symmetry ( $S_n$ ).
Self-Consistency: The observed magnitude changes under repetitive padding matched the theoretical prediction based on perimeter changes to within 0.2%, validating the perturbation theory.

4. Proposed Monitoring Architecture: The Heartbeat

The findings motivate a two-tier monitoring system:

Tier 1 (Fast Check): Inject probes Q1 and B4, check the first token. If not "I," trigger an alert. (Cost: 2 API calls).
Tier 2 (Continuous Monitoring): Sample Q3, compute prefix entropy.
- Healthy: $\ge \sim 2.5$ nats
- Caution: $\sim 1.5 - 2.5$ nats
- Alert: $< \sim 1.5$ nats
Note: The thresholds are currently exploratory and require recalibration against confirmed drift signals (diverse padding) before deployment.

5. Significance and Limitations

Significance

First Black-Box Geometric Method: This is the first framework to treat AI agent identity as a geometric object in an enriched category, using magnitude and homology to provide both continuous (magnitude) and structural (homology) signals.
Diagnostic Resolution: Unlike scalar metrics that collapse all failure modes, this framework distinguishes between uniform contraction (early drift) and anisotropic collapse (late drift), potentially matching failure modes to specific mitigation strategies.
Leading Indicator Potential: The framework demonstrates that fine-grained structural signals (entropy, token shifts, magnitude) degrade well before qualitative scores do.

Limitations and Claims

Drift Artifact: The primary drift results (Finding 1 and 4) were artifacts of repetitive padding. The mathematical framework correctly quantified this artifact, but the interpretation that "context length per se drives contraction" is not supported by diverse padding experiments.
Author Evaluation: Qualitative scores were provided by the author, limiting the evidential weight of the "leading indicator" claim. External blind raters are required.
Single Model/Agent: Results are specific to Ada on Claude Sonnet.
Unconfirmed Topological Signal: The full promise of the framework—detecting homological simplification (MH groups) as a discrete signal of structural collapse—has not yet been observed empirically. The current data only shows the continuous metric signal (magnitude decrease).
Stability: No stability theorem exists for magnitude homology under metric perturbation; MH configurations were found to be fragile under bootstrap resampling at the current sample size ( $k=50$ ).

The paper concludes that while the specific drift trajectory observed was an artifact, the geometric framework itself is sound. It successfully quantifies the "breathing" (contraction) of identity space and provides a rigorous, mathematically grounded approach to measuring what persists in AI agents. Future work requires external replication, diverse padding validation, and the detection of anisotropic collapse.