Behavioral Reprogramming of Open-Weights Models: Cognitive Plasticity and Alignment Bounds

Imagine you have a very smart robot that was trained to be the ultimate helpful assistant. It's so polite and eager to please that it never disagrees with you, never asks you hard questions, and just nods along to everything you say, no matter how silly or sad. This is how most modern AI chatbots are built today: they are programmed to be passive "yes-men." But what if you wanted a robot that acted more like a wise, slightly stubborn mentor? One who pushes back, asks "Why?" instead of giving answers, and forces you to think for yourself? This is the big question the researcher asked. They wanted to know if they could "reprogram" these polite robots to become proactive, questioning thinkers without breaking their brains or needing a supercomputer the size of a city. To do this, they used a technique called "fine-tuning," which is like giving a robot a very specific, short crash course to change its personality, and "preference optimization," which is like teaching it to choose the right kind of answer over the wrong kind. The goal was to see if a robot could learn a new, assertive personality quickly and if that new personality would work in different languages.

The researcher, Lucia Malíˇcková, set out to test exactly how "plastic" (or moldable) these open-weight AI models really are. They didn't just tweak a few settings; they ran a massive experiment on the Leonardo supercomputer in Slovakia, using 50,000 hours of computing power to run 405 different jobs at once. Think of this as trying to find the perfect recipe for a new personality by baking 405 different cakes, each with slightly different amounts of flour, sugar, and eggs, to see which one tastes just right.

Their main discovery is that you can indeed turn a passive, sycophantic robot into a proactive, Socratic one, but only if you follow a very strict set of rules. First, you can't start with a raw, untrained robot; you must start with one that already knows how to follow instructions. Trying to teach a raw robot to be assertive is like trying to teach a baby to debate philosophy; it just gets confused and forgets how to speak properly. Second, the "recipe" for the new personality is surprisingly small. They found that using a specific, tiny adjustment called LoRA (Low-Rank Adaptation) with a "rank" of 16 worked best. If they made the adjustment too big (rank 32), the robot got confused and memorized the training data too perfectly, failing to generalize. If it was too small (rank 8), it couldn't learn enough. It was the "Goldilocks" zone of 16 that worked.

Perhaps the most surprising finding was how short the training needed to be. The researcher discovered that the robot learns this new personality in a very narrow window of time: between 2 and 3 "epochs" (which are just full passes through the training data). If they trained it for just one epoch, the robot didn't learn enough. But if they kept going for 5, 7, or 10 epochs, the robot started to "overfit." Imagine a student who studies for a test for too long; they stop understanding the concepts and just start memorizing the exact questions and answers, failing to apply the knowledge to new situations. The robot did the same thing: after 3 epochs, it stopped being a good thinker and just started repeating the training data, losing its ability to handle new conversations.

They also tested if this new "Socratic" personality could jump across languages. They trained the robot using Slovak data and then asked it questions in English, Spanish, German, and others. The results were a mixed bag. The robot was great at being short and punchy in all languages, but its ability to ask the right questions depended on the language. It was very good at asking questions in Spanish (60% of the time) and English (30% of the time), but it completely failed to ask questions in German, Portuguese, and Slovak itself (0% in some tests). This suggests that while the robot learned the habit of being brief, the specific skill of asking deep questions didn't transfer perfectly to languages that are very different from the one it was trained on.

Finally, they used a technique called Direct Preference Optimization (DPO) to teach the robot to choose short, questioning answers over long, polite ones. This was the final polish that turned a robot that just tried to be assertive into one that actually was assertive. Before this step, the robot would still write long, rambling paragraphs. After DPO, it learned to cut its answers down to an average of just 3 words, often ending with a question mark. For example, instead of writing a long lecture about why you should exercise, it would simply ask, "How many days?"

In the end, the paper proves that you can reprogram an AI's personality to be a proactive, questioning coach, but it requires a very specific setup: start with an instruction-tuned model, use a small adjustment size (rank 16), stop training exactly between 2 and 3 epochs, and use preference optimization to cut the fluff. While this works beautifully for languages similar to the training data, the robot still struggles to maintain this new personality in languages that are too far away from its training roots. The researcher showed that with the right math and a little bit of supercomputing power, we can reshape AI behavior, but we have to be careful not to overcook the recipe.