The Secret Life of AI: When Politeness Meets Code

Imagine you are talking to a giant, super-smart library that has read almost everything ever written. This library is an Artificial Intelligence, or "AI," and it's famous for its ability to chat, tell jokes, and write stories. But here's the catch: while humans love to chat in messy, flowing sentences, computers don't speak that way. Computers speak in strict, organized boxes called "structured data," like JSON or XML. Think of it like the difference between a poet writing a free-form sonnet and a cashier scanning items into a register where every item must fit a specific slot.

For a long time, scientists have studied how these AIs behave when they are chatting like poets. They ask questions like, "Name a tree," and see what the AI says. Usually, if you ask 44 different AIs this question, they might all pick different trees, or they might all pick the same one by accident. This paper asks a simple but tricky question: What happens when we stop asking the AI to chat like a poet and force it to answer like a cashier? Does the AI change its mind? Does it become more boring? Or does it stay the same? This matters because the AI you talk to on your phone might be very different from the AI that actually runs the software in the background, and we need to know if that difference changes how the world works.

---

The Great AI Personality Shift

In this study, researchers took a famous test called the "One-Word Census." Imagine a game show where 44 different AI models are asked 31 questions, like "Name a tree" or "Pick a word." In the original game, the AIs just answered in plain English. The researchers found that even without rules, the AIs tended to agree on certain answers (like "oak" for a tree), but they still had some personality and variety.

Then, the researchers changed the rules. They didn't change the questions, and they didn't use any special computer tricks to force the answers. They simply added one tiny instruction to the end of the prompt: "Reply with JSON only."

JSON is a way of writing data that looks like a little box with a label, such as {"word": "oak"}. It's the language software uses to talk to other software. The researchers wanted to see if just asking the AI to speak in this format would change what it decided to say.

The Big Discovery: The "Format Tax"

The results were surprising. When the AIs were forced to reply in JSON, they became much more alike. The variety of answers dropped sharply.

• Before the change: In plain chat, the most common answer (the "mode") was chosen about 41% of the time.
• After the change: In JSON, that same answer jumped to 64% of the time.
• The variety: The number of unique words the AIs came up with dropped from 52 different words down to just 36.

The researchers call this a "progressive format tax." It's like a tax on being unique. The AIs that were the most creative and unique in plain chat lost the most of their personality when asked to speak in JSON. One specific AI, which was the most "exploratory" (the most likely to pick unusual answers), lost 1.31 bits of its distinctiveness. That's like half of its unique character vanishing just because it had to put its answer in a box.

It's Not a Glitch, It's a "Register"

You might think the AI got confused or that the computer forced it to pick the same answer. But the study ruled that out.

• No "Cooling" Trick: Sometimes, when people want an AI to be less random, they turn down a "temperature" knob. The researchers checked and found the "temperature" didn't change. The AIs were still just as random in their thinking; they just chose different random things.
• Not a Decoding Error: They also tested if the computer was forcing the answer by blocking other words. They found that even without the computer blocking anything, just asking for JSON made the AI change its mind. The change happens in the AI's brain (its "response to the register"), not in the computer's mouth (the decoder).

Think of it like a person who acts differently at a party versus at a job interview. At the party (plain chat), they might tell wild stories and pick weird words. At the job interview (JSON), they suddenly become very professional and pick the "safe," standard answers. The AI isn't broken; it's just "code-switching."

The "Sharpener," Not the "Re-Indexer"

The study found that the AI didn't start picking new answers. It just doubled down on the answers it already liked.

• In 28 out of 31 categories, the most popular answer in plain chat was also the most popular answer in JSON.
• The few times the answer changed (like switching from "chess" to "Monopoly" for a board game), it happened because the original favorite was weak. The JSON format acted like a "sharpener," making the AI's favorite choice even stronger, rather than changing the choice entirely.

The "Personality" is in the Format

One of the coolest findings is that an AI's "personality" depends on the format.

• Some AIs have a "default" answer they love. For example, one AI loved to say "gouda" for cheese in plain chat. But when asked for JSON, it stopped saying "gouda" and went back to the crowd-pleaser.
• Conversely, some AIs developed new defaults just for JSON. One AI said "cerulean" for the color blue 0% of the time in chat, but 100% of the time in JSON.
• This means an AI doesn't have just one personality. It has a "chat personality" and a "JSON personality." They can be totally different.

Why Does This Happen?

The researchers tested different formats to see what caused the change.

• JSON and XML: These formats, which are used for tools and software, caused the AIs to collapse into the same answers.
• YAML and CSV: These are also data formats, but the AIs didn't change their answers as much.
• Brackets: When asked to just put the answer in square brackets [answer], the AIs actually became more diverse.

This suggests the change isn't just about "structure." It's about the specific formats that AIs are trained to use for tools. Since AIs are trained to use JSON to talk to software tools, asking for JSON triggers a "tool mode" that makes them more conservative and uniform.

What This Means for You

The paper concludes with a practical warning. We usually judge AI models by how they chat with us in plain English. We think the "best" AI is the one that gives the most interesting, diverse answers in a chat window. But this study shows that when that same AI is actually used by software (which almost always requires JSON), it becomes much more boring and uniform.

If you are building an app that uses AI to recommend movies, pick a tree, or brainstorm ideas, the AI you see in the chat window might be much more creative than the AI actually running your app. The "JSON version" of the AI is measurably less diverse. The gap between the AI we talk to and the AI that does the work is a fixed cost of the format, not a rare accident.

In short: If you want to know what an AI really thinks, don't just ask it to chat. Ask it to fill out a form. You might be surprised by how much more boring it gets.