LLM Enhancement with Domain Expert Mental Model to Reduce LLM Hallucination with Causal Prompt Engineering

In the world of artificial intelligence, large language models have become remarkably skilled at reading, writing, and summarizing vast amounts of information. They can explain complex scientific theories, draft legal documents, and even write code. However, these systems have a fundamental blind spot: they can only work with knowledge that has already been written down and fed into their training data. When faced with a decision that relies on unwritten intuition, specific organizational context, or a unique combination of circumstances that has never been recorded, the model does not simply admit it does not know. Instead, it often invents a confident but false answer, a phenomenon known as hallucination. This is not a glitch that can be fixed by searching for more documents; it is a structural limit. If the knowledge does not exist in a database, the machine cannot find it. For high-stakes decisions in fields like medicine, national security, or research funding, this gap between what the machine knows and what a human expert knows can be dangerous.

Researchers Boris Kovalerchuk and Brent D. Fegley have proposed a new way to bridge this gap. Rather than trying to make the artificial intelligence smarter or give it more books to read, they suggest building a specific, structured map of how a human expert actually thinks. They call this an Expert Mental Model. The core idea is to stop asking the machine to guess the answer and instead use the machine as a tool to help the expert articulate their own decision-making logic. By turning a human's intuitive judgment into a clear, step-by-step set of rules, the system can then apply those rules to new situations without ever needing to hallucinate. This approach treats the problem not as a failure of the machine's memory, but as a failure to capture the human's specific reasoning process.

The researchers tested this idea across three very different fields: deciding whether to apply for a research grant, designing a cybersecurity system, and making clinical diagnoses. In the first scenario, a team of researchers needed to decide if they should spend months writing a proposal for a funding agency. The decision depended on many factors, such as whether the team had the right skills, if the project fit the agency's goals, and if they had enough time. A standard artificial intelligence could generate a list of questions to ask, but it could not determine how to weigh those answers. If the team said "yes" to having the skills but "no" to having the budget, the machine might guess that the project was still viable, or it might guess it was not. The machine had no way of knowing that for this specific team, the budget was a deal-breaker, while for another team, it might be a minor hurdle.

To solve this, the researchers used a method that relies on a simple principle: if a situation is good enough to say "yes," then making any part of it better should not turn the answer into "no." This is called monotonicity. Imagine a team of experts answering a series of yes-or-no questions about a project. If they decide that a project with a great idea but no money is a "no," then a project with a great idea, no money, and also no team members must also be a "no." By using this logic, the researchers could ask the expert far fewer questions than would normally be required. Instead of asking about every possible combination of factors—which could number in the millions for a complex decision—they asked a carefully chosen set of questions. The answers to these questions allowed them to infer the answers to thousands of other scenarios automatically.

In the grant proposal example, the team started with twenty potential questions. An artificial intelligence helped organize these into a hierarchy, grouping related questions together. Then, the human expert answered a small number of generalized questions. For instance, instead of answering every single detail about resources, the expert might answer a broad question about whether the team was ready. If the answer was "no," the system knew immediately that no further details were needed to reject the project. If the answer was "yes," the system would drill down into the specific details. Through just four rounds of questioning, the researchers were able to map out the expert's decision logic for over 60 percent of all possible scenarios. The remaining scenarios were clustered around the difficult edge cases where the decision was unclear, which is exactly where human judgment is most valuable.

The researchers also tested this method in cybersecurity, where the rules are often defined by government standards. In this case, the artificial intelligence could extract the standard rules from a document, but it still needed a human expert to apply those rules to the specific data of a particular organization. The system helped the expert determine the impact of a security breach on their specific network, translating vague concerns into a structured model. In the third area, healthcare, the researchers reviewed past studies where similar methods had been used to diagnose breast cancer. They found that when the decision rules were extracted directly from radiologists using this structured approach, the resulting models were more accurate and consistent than those built purely from data. The models could explain why a decision was made, tracing it back to specific factors, which is crucial for doctors who need to trust the system.

A critical part of this work was measuring how often the artificial intelligence would get it wrong if left to its own devices. The researchers created simulations where they compared the decisions of a human expert against the guesses of an artificial intelligence. They found that while the overall error rate might look small, the mistakes were concentrated in the most important cases. When the expert would have said "yes" to a project, the artificial intelligence often said "no," missing the opportunity entirely. In some simulations, the error rate for these missed opportunities approached 100 percent. This revealed that the danger of hallucination is not just about making random mistakes; it is about systematically failing to recognize the rare, high-value cases that require specific, unwritten knowledge.

The researchers argue that the solution is not to try to make the artificial intelligence more like a human, but to use the artificial intelligence to help humans encode their own thinking. The machine is excellent at generating lists of factors, organizing them into logical groups, and checking for consistency. It is terrible at knowing what matters to a specific team or organization. By combining the two, the system creates a decision support tool that is both computationally efficient and grounded in real human expertise. The artificial intelligence handles the heavy lifting of structure and organization, while the human provides the specific values and logic that make the decision meaningful.

This approach also helps prevent a subtle risk known as cognitive homogenization, where everyone starts making decisions the same way because they are all using the same artificial intelligence. If an organization relies solely on a generic model, they might lose their unique perspective and the specific insights that come from their own experience. By building their own Expert Mental Model, an organization preserves its distinct way of thinking. The model becomes a portable record of their collective wisdom, surviving even if the experts who created it leave the organization. It ensures that the decision-making process remains transparent and accountable, rather than a black box that produces answers no one can explain.

The study does not claim to have solved every problem. The method works best when the decision logic follows the principle that improving a situation never makes the outcome worse. It also requires that the experts are willing to spend time articulating their thoughts, which can be a challenge. In some cases, experts might struggle to answer abstract questions without seeing the specific details, and the system needs to be flexible enough to handle that uncertainty. Furthermore, while the method has been tested in simulations and reviewed against past studies, the researchers acknowledge that more real-world testing is needed to see how it performs in live clinical settings or with actual grant proposals.

Despite these limitations, the findings offer a clear path forward for using artificial intelligence in critical fields. The key insight is that the machine should not be the decision-maker. Instead, it should be the architect of the decision-making process, helping to build a structure that captures human judgment accurately. By doing so, it avoids the trap of hallucination and provides a tool that is both powerful and trustworthy. The result is a system that does not replace the expert but amplifies their ability to make sound, consistent, and explainable decisions, even in the face of complex and uncertain situations.