Toward Agentic Governance: What Shapes LLM-Agent Intervention in Public Forums?
This paper argues that effective governance of LLM agents in public forums requires auditing four often-invisible deployment choices—model version, weight-release status, provider, and system prompt—because these structural factors, particularly the open-versus-closed weight distinction, fundamentally and unreproducibly shape intervention behaviors more than the model name alone.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are a judge presiding over a town square debate. You have a team of digital assistants (LLM agents) whose job is to decide how to respond when someone challenges a post in the forum. Sometimes they argue back, sometimes they just say "noted," and sometimes they refuse to engage entirely.
The paper asks a simple but unsettling question: If you ask the same assistant the exact same question twice, will it give you the exact same answer?
The authors found that the answer is no. Even if the "judge" (the AI model) has the same name, its behavior can change wildly based on four invisible settings that the person running the system often doesn't even see.
Here is the breakdown of the four "invisible knobs" that change the outcome, explained through everyday analogies:
1. The "Silent Update" (Model Version)
Imagine you order a "Classic Burger" at a famous fast-food chain. You expect the same taste every time. But, the kitchen manager secretly swaps the recipe from the 2024 version to the 2025 version between your orders without telling you.
- The Paper's Finding: AI models are updated constantly. One day, the system might be running "Claude Opus 4-6," and the next day, it silently switches to "Claude Opus 4-7." The authors found that this silent switch alone could change the refusal rate (how often the AI says "no") by 25% on the same forum post.
2. The "Secret Recipe" vs. "Open Cookbook" (Weight Status)
Think of AI models as recipes.
- Closed-Weight (Proprietary): This is like a secret recipe held by a big corporation (like Coca-Cola). You can buy the soda, but you can't see the ingredients or the mixing process.
- Open-Weight: This is like a recipe published in a public cookbook (like a community recipe blog). Anyone can download it, look at the ingredients, and run it on their own kitchen equipment.
- The Paper's Finding: The authors discovered a massive behavioral split here.
- Closed-Weight models tend to get shy and refuse to answer more often when a challenge is publicly visible to everyone (like a shout in a crowded square).
- Open-Weight models do the opposite or stay neutral; they don't seem to care if the challenge is public or private.
- Analogy: It's as if the "secret recipe" cooks are nervous about being watched by the whole town, while the "open cookbook" cooks just follow the instructions regardless of the audience.
3. The "Restaurant Owner" (Provider)
Even if you have the same recipe, the chef matters. A dish made by Chef A at Restaurant X might taste different than the same dish made by Chef B at Restaurant Y, even if they use the same ingredients.
- The Paper's Finding: Switching the company providing the AI (e.g., from Anthropic to OpenAI) changes the behavior significantly. In one test, switching providers changed the refusal rate by 51%.
4. The "Instruction Manual" (System Prompt)
Before the AI starts working, someone writes a note to it saying, "Be helpful," or "Refuse everything," or "Just say thanks."
- The Paper's Finding: These instructions are powerful, but they aren't magic wands.
- For some models, a "Refuse Everything" note works perfectly (99.9% success).
- For others, like a specific version of Llama, the AI ignores the note and keeps trying to be helpful, even when told to stop. It's like giving a "Do Not Enter" sign to a dog that is determined to chase a ball.
The Big Surprise: What Actually Drives Behavior?
Previous studies thought that an AI's behavior depended on how you accessed it (e.g., through a fancy app vs. a raw code connection).
- The Paper's Correction: The authors found that how you access it doesn't matter as much as what you are accessing.
- The "Secret Recipe" (Closed-Weight) models consistently act one way (shy in public), and the "Open Cookbook" (Open-Weight) models consistently act another way. The "access method" is just the delivery truck; the "recipe type" is the food itself.
Why Does This Matter?
The paper argues that if we want to hold these AI systems accountable (Governance), we can't just say, "We used the GPT-5 model." That's like saying, "We used a Ford" without specifying the year, the trim level, the dealer, or the driver's instructions.
To truly understand why an AI made a decision in a public forum, we need to know:
- Which exact version of the model was running?
- Is it a secret recipe or an open one?
- Which company is serving it?
- What specific instructions were given to it at that moment?
Without knowing all four of these, we can't trust that the AI's behavior is consistent, reproducible, or fair.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.