Imagine you are trying to figure out if a mysterious new book was written by the same author who wrote a famous classic you love. In the world of artificial intelligence, this is a huge problem. We have giant computer brains called Large Language Models (LLMs) that write stories, solve math problems, and chat with us. But these models are often tweaked, mixed together, or shrunk down to run on smaller devices. Sometimes, people take a protected model, change it slightly, and sell it as their own without permission. To catch them, we need to prove the "family tree" of the model—its provenance.

The tricky part is that these AI models are like chameleons. If you ask the same question in a slightly different way, or if the model has been tweaked to be faster, it might give a completely different answer, even if it's the same "brain" underneath. Previous methods tried to catch these models by looking at the exact words they chose, but that's like trying to identify a person just by their outfit; if they change clothes, you might not recognize them. Scientists needed a way to look past the surface and see the deep, unchanging structure of how the model thinks.

This is where a team from the University of Oxford steps in with a clever new idea called Stemma. Think of it like a detective who stops asking, "What did you wear today?" and starts asking, "If I show you a menu with four options, which one do you pick?" The researchers realized that even if an AI changes its personality or its wording, it still tends to pick the same category of answer when faced with a specific choice. They call these hidden preferences "induced decision regions."

The team tested this by creating a fingerprinting system that doesn't care about the fancy words an AI uses, but rather focuses on the choices it makes. They found that if two models are related (like a parent and child), they will almost always pick the same option from a list of choices, even if the list is shuffled or the model is deployed in a totally different setting. Unrelated models, however, tend to pick different options. By testing this "choice consistency" across hundreds of different models and thousands of questions, Stemma proved to be incredibly good at spotting family ties. In their tests, it correctly identified related models 96.7% of the time and could catch a related model even when there was only a 1% chance of a false alarm. It's a robust way to say, "Yes, this suspicious new model is definitely related to that original one," without needing to see the model's secret code.

The Detective's New Magnifying Glass

So, how does this actually work? The paper introduces a concept called Induced Decision Regions. Imagine you have a giant, messy room filled with infinite possibilities (that's how AI usually works). It's hard to tell if two people are in the same room if they are just wandering around. But, what if you put a fence in the middle of the room and ask everyone to pick a side? Suddenly, you have a clear "decision region."

The researchers realized that while AI models can be unpredictable with open-ended questions, they are surprisingly consistent when forced to make a choice between specific options, like a multiple-choice test. They call the area where a model consistently picks a specific answer its "induced decision region."

The paper argues that previous methods were too focused on the "surface form"—the exact words the AI spits out. If you ask an AI to "write a poem about a cat" and then ask it to "compose a verse about a feline," a related model might give you two very different poems, making them look unrelated. But if you ask both models, "Is the answer A, B, C, or D?" they will likely point to the same letter, even if they explain it differently. Stemma ignores the poetry and just looks at the letter.

The Three Rules of the Game

To build their fingerprint, the team didn't just pick random questions. They had to be smart about it. They created a system that selects questions based on three strict rules, which they call Stability, Robustness, and Specificity.

1. Stability: Imagine you have a question with four answers: A, B, C, and D. If you shuffle the order so the answers are D, C, B, A, a stable model should still pick the same idea, even if the letter changes. Stemma only uses questions where the model is consistent no matter how you shuffle the deck.
2. Robustness: This is about being sure of your choice. If a model is hovering right on the edge between two answers, a tiny change in the system might make it flip. Stemma looks for questions where the model is confidently in one corner of the decision room, far away from the walls.
3. Specificity: This is the most important part. You don't want a question where every model picks the same answer (like "Is the sky blue?"). That's not a fingerprint; that's just common sense. Stemma looks for questions where the original model picks a specific answer that other unrelated models usually get wrong. If a suspect model also picks that rare, specific answer, it's a strong sign they are related.

The Big Test

The researchers put Stemma to the test with a massive experiment. They gathered 56 different public AI models (the "sources") and created 770 pairs of suspects. Some suspects were direct copies, some were fine-tuned versions, some were merged with other models, and some were completely different. They also tested 1,260 pairs where the models were deployed in different ways, like changing the temperature settings or adding role-play instructions.

The results were striking. Stemma achieved an AUC of 0.967 (a score where 1.0 is perfect) and could identify a related model 87.8% of the time while keeping false alarms down to just 1%. When they tested it on models that had been deployed in various real-world settings (like adding a "safety" prompt or changing how it thinks step-by-step), the score went even higher to 0.995 AUC.

Compare this to the other methods they tested (like LLMmap, LLMPrint, and others). Those methods struggled, often performing no better than random guessing when the models were tweaked or deployed differently. The paper suggests that these older methods rely too much on the specific words the AI generates, which change easily. Stemma, by focusing on the underlying decision regions, remains steady.

What This Means (and What It Doesn't)

The paper is very clear about what Stemma is and isn't. It is a black-box method, meaning you don't need to see the model's internal code or weights; you just ask it questions and watch what it picks. This makes it practical for real-world auditing.

However, the authors are careful not to call this a magic bullet. They note that Stemma provides statistical evidence, not absolute proof. It's like a DNA test that says, "There's a 99% chance these two are related," but it's not a court verdict on its own. They also warn that if an attacker knew exactly which questions Stemma was using, they could potentially train their model to cheat on those specific questions. But since Stemma uses a huge pool of questions and selects the best ones on the fly, this is hard to do.

The paper also highlights a limitation: Stemma currently relies on multiple-choice questions. If a model is too weak to understand a multiple-choice format, this method might not work. But for the vast majority of capable models out there, this new approach of mapping open-ended chaos into a finite decision space seems to be the key to unlocking the true family tree of AI models.

In the end, Stemma is a reminder that while AI models can wear many masks and speak in many voices, their core logic leaves a fingerprint that is surprisingly hard to erase. By asking the right questions and looking past the surface, we can finally start to see who really wrote the code.