In the world of artificial intelligence, large language models have become remarkably fluent at writing text. They can answer questions, summarize stories, and hold conversations by predicting the next word in a sentence based on patterns they learned from vast amounts of data. However, these models have a fundamental weakness: they do not know the difference between a fact that is true and a sentence that merely sounds plausible. To fix this, engineers developed a method called retrieval-augmented generation. This approach gives the model a set of documents to read before it answers, forcing it to ground its response in real evidence rather than just its own memory. Yet, a new problem arises with this method. When the system searches for evidence, it often finds a paragraph of text where the answer is buried inside a sentence. The model must then act like a detective, pulling the specific fact out of the prose. In doing so, it can easily misread the context or reconstruct the value incorrectly, leading to answers that are fluent but subtly wrong.

To solve this, researchers at the University of Salerno in Italy built a new system called ORACOLO. They wanted to know if they could make the search for facts more precise by using a formal map of knowledge, known as an ontology, instead of just searching through plain text. An ontology is a structured database where facts are stored as clear, isolated connections, like a library card catalog where every book is listed with its author and year in a rigid format. The challenge is that these catalogs are difficult for regular people to query because they require a specialized, formal language. The team combined the best of both worlds: they used a neural network, a type of artificial intelligence good at understanding human language, to translate a user's question into a formal query. Then, they used a symbolic engine, a strict computer program, to execute that query against the structured map. If the map returned a clear answer, the system used it. If the map returned nothing, the system fell back to the traditional method of searching through text paragraphs. This hybrid approach allowed them to test whether the structured map could provide more accurate facts than the text search, even when both methods used the same language model to write the final answer.

The researchers tested this system on a real-world library catalog from the University Library of Lugano in Switzerland. This catalog contained nearly one hundred thousand facts about books, including titles, authors, publication dates, and summaries of the content. They created a set of questions based on this catalog, ranging from simple requests like "Who wrote this book?" to more complex chains of information. They then compared their new system against a standard version that only used text search, ensuring that both systems used the exact same language model to generate the final answer. This was a crucial step, as it meant any difference in the results had to come from how the information was found, not from the intelligence of the writer. The results showed a clear improvement in accuracy. The new system correctly identified the specific facts in the answer much more often than the text-only system. While the text-only system managed to get the right answer about twenty-nine percent of the time, the new system improved this to nearly forty-seven percent.

The study revealed that this improvement came specifically from the ability to retrieve exact facts. When the question asked for a single piece of information, such as an author's name or a publication year, the new system was significantly better. It also performed well when the question required linking two pieces of information together. However, the advantage disappeared when the question required the system to reason about the facts, such as inferring a theme from a book's content. In those cases, both systems performed similarly. This suggests that the new architecture does not make the computer smarter at thinking; rather, it makes the computer better at finding the exact raw materials it needs to think with. The system also proved that the structured map did not make the answers more "faithful" to the evidence in a way that the text search did not. Both systems were equally good at sticking to the information they were given. The difference was simply that the new system was given better, more precise information to begin with.

A key finding of the research was how the system handled its own mistakes. The researchers discovered that the formal map did not always work. In about twenty-four percent of the cases, the translation from human language to the formal query failed, or the query returned no results. In these moments, the system did not give up. It detected the failure and immediately switched to the text search method to find an answer. This safety net meant the system could still provide a response even when the strict map could not. However, the researchers also found that the system was most accurate when the formal map worked correctly. When the system relied on the text fallback, the improvement in accuracy was much smaller. This confirmed that the structured map was the source of the precision, while the text search served as a reliable backup.

The study also uncovered a surprising issue with how we measure the quality of these systems. One common metric used to evaluate answers is "relevance," which checks how well the answer matches the topic of the question. The researchers found that a system that made no factual errors but simply guessed fluently could score higher on relevance than their accurate system. This happened because the guessing system wrote long, flowing answers that sounded perfect, while the accurate system sometimes gave short, direct answers or declined to answer when it could not find the facts. This suggests that relevance scores can be misleading, rewarding style over substance. The researchers concluded that to truly evaluate these systems, we must look at whether the facts are correct, not just whether the answer sounds good.

Finally, the team examined the cost of running this new system. The process of translating a question, checking it against the map, and then writing the answer took significantly longer than simply asking the language model to guess. On average, the new system took about 292 seconds to answer a single question, compared to about thirty-five seconds for the text-only system. The researchers noted that this delay was mostly due to the time it took for the language models to process the information, not the time it took to check the map. They found that one specific step in their process, which tried to break complex questions into smaller parts, was actually slowing the system down without improving the results. Removing this step cut the time significantly without hurting the accuracy. This suggests that while the system is currently too slow for instant, interactive use, it could be practical for situations where accuracy is more important than speed, such as verifying historical records or checking institutional data. The work demonstrates that by combining the flexibility of human language with the precision of structured data, we can build systems that are far more reliable than those that rely on language alone.