Gates Instead of Prompts: Development and Methods of a Large-Language-Model Expert System for Root- Cause Analysis in a Steel Foundry

In the heavy industry of steel casting, where molten metal is poured into sand molds to create massive components, the difference between a perfect part and a defective one often comes down to invisible physics. When a casting fails, engineers must perform a root-cause analysis, a rigorous investigation that pieces together clues from machine logs, laboratory tests, and complex computer simulations of how the metal cools and solidifies. For decades, this work has relied on human experts who can read thousands of pages of manuals and interpret subtle patterns in data. Recently, a new tool has entered the conversation: large language models. These are powerful computer programs capable of reading vast amounts of text and answering questions with remarkable fluency. However, they carry a significant flaw: they speak with absolute confidence even when they are wrong. In a high-stakes environment like a steel foundry, a confident but incorrect answer could lead to dangerous mistakes or costly production errors. The challenge, therefore, is not just to use these smart computers, but to build a system that forces them to be honest, grounded, and verifiable.

This paper describes the creation of such a system, developed by an independent researcher for a steel foundry in the Czech Republic. The project, named Digital Foundry PRO, was not designed to replace human engineers but to act as a forensic assistant that works offline on historical data. The core discovery of the work is that you cannot simply tell a large language model to "be careful" or "check your facts." Instead, the researchers had to build a rigid framework of digital gates that physically stop the computer from making unsupported claims. The system was tested on real-world data from a foundry that produces steel castings using green sand molds. Over six months, the system evolved from a simple prototype into a sophisticated tool that could analyze casting defects and equipment failures. The most important finding was that the system's ability to find the true cause of a problem did not come from giving the computer more tools or more data, but from changing the specific instructions it was given. By forcing the computer to look at the exact moment and place where a defect forms, rather than scanning the whole simulation for anything unusual, the system's accuracy improved dramatically.

The development process began with a stark realization about the limitations of current technology. In one early test, a standard chatbot was asked why a pouring unit was stopping its cycle. The machine looked at the temperature data, assumed the metal was aluminum, and declared that the recorded temperatures of over 1,300 degrees Celsius were physically impossible. It then recommended replacing the sensor and lowering the alarm settings. The advice was confident and sounded reasonable, but it was completely wrong because the foundry was pouring steel, not aluminum. This error highlighted the danger of a system that sounds authoritative but lacks a mechanism to verify its own facts. To solve this, the researcher built a series of "gates" into the software. These are not suggestions for the computer to follow; they are hard-coded rules that stop the process if certain conditions are not met. For instance, the system will not start an analysis unless the specific type of metal is confirmed by a document in its library. If the computer tries to cite a page from a manual, the software checks the actual text of that page to ensure the quote is there word-for-word. If the computer tries to reference a simulation image, the software verifies that the image file actually exists and matches the description. These gates act as a safety net, ensuring that the computer cannot invent facts or hallucinate evidence.

The system also introduced a new way of reading computer simulations of the cooling process. Usually, a computer might scan an entire simulation video looking for any sign of trouble, such as a hot spot or a fast-moving stream of metal. This approach often leads to false alarms, as high temperatures or fast flows are sometimes normal parts of the process. The new system changed the rules: it only looks at the simulation data at the exact location, at the exact moment, and above the exact threshold where a specific defect is known to form. For example, if a defect is a type of shrinkage that happens near the core of the casting, the system ignores the rest of the mold and focuses only on that specific area during the specific time the metal is solidifying. This method prevents the computer from getting distracted by irrelevant data. Furthermore, the system treats the manuals for the machinery with extreme caution. A manual for a piece of equipment often describes every possible feature the machine could have, not just the features the specific machine in the factory actually possesses. The system was programmed to assign a "trust level" to every claim about the equipment. Only claims that were verified against the actual machine's operating data were allowed to be used in official procedures. Claims based solely on the manual were flagged as unverified, preventing the system from recommending actions that the specific machine could not perform.

The most significant breakthrough came when the researchers tested how to make the system think deeper. Initially, the computer could read the data and produce a report, but the reports were shallow and often missed the actual problem. The researchers tried adding more tools and more data, but the results did not improve. They then changed the task itself. They gave the computer five specific instructions, or "mandates," that forced it to look at the problem differently. These instructions told the computer to examine the area around the cores of the casting, to separate the central hot spot from the surrounding zones, and to actively try to disprove simple explanations. With these new instructions, and without adding any new tools, the system's performance transformed. In a series of tests on a specific steel casting, the number of times the system correctly identified the actual location of the defect jumped from zero to between three and sixteen mentions in the same report. The computer began to find the real problem, which was a subtle type of porosity in a ring around the cores, rather than the obvious textbook answer of a large shrinkage hole. It is important to note that this improvement was measured against a specific reference analysis; the reference analysis was written by an LLM under the author's guidance; it is not an independent expert report. This proved that the depth of the analysis depended on the quality of the question asked, not the quantity of data provided.

Throughout the project, the researchers also documented what did not work, which was just as important as the successes. They built a "digital twin," a virtual model of the entire production line intended to predict failures in real-time. However, they discovered that the data feeding this model was flawed. The system was using limits and values that had been guessed or taken from a different machine, leading it to flag almost every measurement as an error. The researchers realized that the system was not finding faults in the factory; it was finding faults in its own understanding of the factory. They deliberately dismantled this digital twin, removing the interface and the machine model, because it was not ready for the real world. They also found that a statistical link between higher temperatures and more gas defects was actually an illusion caused by a faulty sensor reading. When they checked the source of the data, they saw that the temperature readings were not reliable, so they retracted the finding. This willingness to tear down their own work and admit when a result was an error became a central part of the system's reliability.

The final result is a system that operates on a simple but powerful principle: the computer proposes, and a human signs. The system is designed to work offline, analyzing historical data that has already been exported from the factory. It does not connect to the live control systems of the machines, and it does not make decisions on its own. Instead, it acts as a highly disciplined assistant that gathers evidence, checks its own facts against strict rules, and presents a report to a human engineer. The human then reviews the findings, verifies the evidence, and takes responsibility for the final decision. The researchers found that the system could identify chronic faults, such as a misaligned axis on a pouring unit that had been causing outages for months, and could distinguish between real defects and data errors. However, they also noted that the system is not perfect. It still struggles with quantifying the exact severity of some defects, and it relies heavily on the quality of the data it is given. If the data is missing or incorrect, the system cannot fix it. As noted previously, the reference analysis used to test the system was written by an LLM under the author's guidance; it is not an independent expert report.

The paper concludes with a clear recommendation for the future of this technology in industry. The researchers suggest that the highest level of autonomy a system like this should ever reach is a "preparatory" level. At this level, the system can prepare a value or a recommendation, check it against safety limits, and wait for a human to confirm it before any action is taken. They explicitly advise against allowing the system to write directly to the control systems of the factory, even if it is fully validated. The responsibility for the final product belongs to the human engineer, and that responsibility should never be handed over to software. The project demonstrates that while large language models are powerful tools for reading and processing information, they must be wrapped in a layer of deterministic, code-enforced rules to be useful in critical industrial settings. The success of the system did not come from making the computer smarter, but from making the rules around it stricter. By forcing the computer to prove its claims, to look at the right place at the right time, and to admit when it does not know the answer, the researchers created a tool that can genuinely assist experts in solving the complex problems of steel casting. The work stands as a practical guide for how to build reliable AI systems in the real world, where the cost of a mistake is not just a wrong answer, but a broken machine or a failed product.