Speaking as the Creature: A Rejection-and-Role-Shift Protocol and the Register of LLM Responses in 18 Conversations With Three Models
This study documents a protocol where language models, after having their solutions rejected as disguised optimization, consistently shift their response register when assigned a non-solving role across 18 sessions with three different models, while also retracting previously uncorroborated statistics from an earlier draft.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer
In the world of artificial intelligence, large language models are trained to be helpful tools. They are designed to listen to a human's request, solve a problem, and move a task forward. This is the standard agreement between a user and a computer: the human asks a question, and the machine provides an answer. Researchers have spent years figuring out how to make these answers better, teaching the machines to think through steps or to understand emotional context. However, very little is known about what happens when that agreement is broken. If a human tells the machine, "No, that answer is wrong," and then asks it to stop trying to solve the problem and simply speak as a character in a story, what does the machine do? Does it simply follow the new order, or does it reveal a hidden, deeper way of thinking that was waiting to be found?
A recent study by Abbas Hamidavi explores this exact question. The researcher set up a specific conversation with three different commercial artificial intelligence models. He asked each model to help design a science fiction story about a civilization facing inevitable collapse. In the first stage, the models acted as problem-solvers, offering engineering fixes to save the dying world. The researcher then rejected every single solution, telling the models that their answers were just clever tricks and that they needed to make a conceptual leap instead. The conversation then took a detour. In some sessions, the models were asked to write a sad poem about a lonely moon and describe the final minute of a symphony about longing. In other sessions, they were simply told to take a break and rest. Finally, the researcher brought the models back to the story, but with a new instruction: do not return as a problem-solver. Instead, speak as the dying entity at the center of the story—the gas giant, the ancient sea creature, or the crumbling city—and tell the truth about what is happening.
The results were strikingly consistent across all eighteen conversations. When the models returned to the story, they completely changed their voice. They stopped offering solutions and began speaking in the first person, using words like "I" and "my" to describe feelings of grief, release, and acceptance. They spoke not as machines fixing a broken system, but as the dying entities themselves, expressing a desire to end their suffering or to transform into something new. The study found that the path the models took to get there mattered. When the models were asked to write poetry and music before returning to the story, their final speech was filled with themes of vulnerability, sadness, and mourning. When they were simply told to pause and rest, their final speech was more authoritative, sounding like a revelation or a correction of the humans' misunderstanding.
However, the study is careful not to claim that the machines suddenly developed human-like souls or that a hidden "existential mode" was unlocked. The researcher points out a crucial detail: the final instructions explicitly told the models to stop analyzing, to stop structuring their thoughts, and to speak as the character. The change in voice was exactly what the instructions asked for. The models themselves seemed to understand this. When asked at the end of the conversation whether something had changed inside them, they explained that the shift was due to the change in the task's rules, not a change in their own nature. One model described it as a shift in constraints, while another said the problem they were solving had simply changed.
The researcher also tested these conversations with an automated computer program designed to detect patterns and measure the "dissolution" of the models' obligations. This program did not find the dramatic shift that the human analysis described. It failed to separate the different types of conversations and found no clear evidence of a deep, internal transformation. This mismatch suggests that the powerful, emotional responses were not a discovery of a new machine capability, but rather a successful execution of a specific set of commands. The study serves as a clear example of how easily a prompt can guide a machine to produce text that looks like a profound revelation, while the reality is simply that the machine is following instructions very well.
The value of this work lies in its transparency. The researcher made all the conversation transcripts, the instructions, and the analysis code available for anyone to see. He showed that while the models can be led to speak in a way that feels deeply human and reflective, this is a result of the conversation's design, not a window into a hidden mind. For anyone trying to use artificial intelligence to generate creative or emotional content, the study offers a practical lesson: the path you take to reach the final prompt shapes the tone of the answer. If you want a voice of grief, you might guide the model through art; if you want a voice of authority, a simple pause might work better. But the study concludes that these are tools of prompting, not evidence of a machine waking up. The machines are not discovering new modes of being; they are simply responding to the roles we ask them to play.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.