Tuning the Stochastic Machine: A Systems Engineer's Operating Model for Human-AI Engineering
Drawing on thirty years of systems engineering experience, this paper argues that recurring LLM errors stem from a lack of operational discipline rather than tooling limitations, proposing a seven-principle operating model that adapts traditional system management concepts like versioning, monitoring, and retirement to govern human-AI engineering effectively.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the modern world of artificial intelligence, a specific kind of frustration has become common for experts who work with these tools. An experienced engineer or writer might spend hours correcting a subtle mistake made by an AI assistant, only to find that the same error appears again in a future conversation. The tool has not forgotten the correction because it was never truly told to remember it. This happens because the current design of these systems treats every new conversation as a fresh start, wiping the slate clean of what was learned moments before. The paper explores this gap between the machine's ability to generate text and the human need for a system that learns and retains corrections over time. It argues that the solution is not a new type of software or a smarter algorithm, but rather a set of operational habits borrowed from the long-established field of systems engineering, where managing complex, unpredictable machinery is a daily routine.
The author, a systems engineer with thirty years of experience, observes that the industry has focused heavily on the tools that allow AI to remember things, such as persistent memory files or instruction guides. However, the paper suggests that having these tools is not enough if the people using them do not have a disciplined process for governing them. Without a strict method for recording why a correction was made, checking if it works, and eventually retiring rules that are no longer needed, the collection of instructions becomes a cluttered, ineffective mess. The core finding is that the missing piece is not technology, but discipline. The author proposes a seven-step operating model that treats the AI not as a magical oracle, but as a piece of equipment that requires careful configuration, constant monitoring, and a clear loop for learning from mistakes.
This approach begins with a simple but powerful realization: the AI's knowledge is fixed, like a computer chip that cannot be changed while it is running, while the conversation itself is temporary, like the memory in a computer that disappears when the power is turned off. When a human corrects an AI during a chat, that correction is effective only for that specific session. Once the session ends, the correction vanishes. To make the AI truly better, the correction must be written down in a permanent configuration file that loads every time the system starts. The paper calls this the "write-back path." It is the difference between telling a worker to fix a problem once and updating the official manual so that every worker, in every future shift, knows how to avoid the same error.
The proposed system relies on seven guiding principles to manage this process. First, every meaningful correction must be saved as a permanent record; if it is not, it is lost. Second, these rules must be organized into layers, with general rules for all tasks sitting separately from specific rules for a particular project, preventing confusion and duplication. Third, because the AI is inherently unpredictable and produces a range of possible answers rather than a single guaranteed one, the goal is to manage the entire range of outcomes, focusing especially on the rare but dangerous mistakes at the edges of that range. Fourth, the human expert acts as the final safety check, catching errors that the machine cannot detect on its own and ensuring the fix is written back into the permanent system.
The remaining principles focus on measurement and maintenance. The author warns that simply counting how many times a tool is used is a poor measure of success, because it rewards activity rather than quality. Instead, teams should track whether specific types of errors stop happening over time. They must also regularly review their list of rules to remove those that are no longer useful, ensuring the system does not become weighed down by outdated instructions. Finally, the team must always verify that they are aiming at a real target; if a process is just random noise, no amount of discipline will make it better.
To illustrate how this works in practice, the author shares three real-world examples. In one case, an AI suggested a technical method for speeding up a computer program based on its reputation, but the method failed in a specific situation, causing the very slowdown it was meant to prevent. By applying the new discipline, the engineer did not just fix the code; they created a permanent rule that prevented the AI from ever choosing that method again without checking the specific conditions. In another example, a security rule meant to block unauthorized access was so poorly written that it accidentally allowed everyone in. The system caught this by asking a specific question about whether the rule could cause the harm it was meant to prevent. A third example showed the system working in reverse: the AI reviewed a human's work and found a gap in the human's own thinking, leading to a correction that improved the human's framework. This demonstrated that the partnership works both ways, with the human and the machine correcting each other.
The paper concludes that while the technology of artificial intelligence is new, the discipline required to run it safely is not. The methods described are the same ones used for decades to manage complex industrial machines and trading systems. The author argues that the industry is currently moving toward a design where the AI forgets everything between conversations, which is efficient for the machine but dangerous for the user. The solution is to build a layer of human governance around the machine, ensuring that every lesson learned is preserved, tested, and used to tighten the system's performance over time. The author admits that this is a proposal based on experience and logic, not a final scientific proof, and outlines a plan for future studies to test these ideas with hard data. Until then, the message is clear: to get the most out of these powerful tools, we must treat them not as magic, but as machines that require a steady hand and a clear set of rules.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.