DocuStore: Domain Constrained Retrieval Augmented Generation for Drug Discovery Documents
The paper introduces DocuStore, a domain-constrained retrieval-augmented generation architecture that integrates molecular structures and biological entities to significantly improve question answering accuracy and evidence retrieval in drug discovery documents, as validated by the new DocQABench benchmark.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the high-stakes world of drug discovery, scientists generate a vast and growing library of documents. These are not just simple reports; they are complex collections of text, data tables, hand-drawn chemical diagrams, and precise measurements of how well a potential medicine works against a disease target. For decades, researchers have relied on these files to remember which compounds they tested, what results they got, and how different molecules relate to one another. However, as the volume of this information has exploded, finding a specific piece of evidence has become increasingly difficult. Traditional computer search tools, which work by matching the general meaning of words, often struggle here. They can find a paragraph that mentions a chemical name, but they might miss the specific number describing that chemical's strength if it appears in a table or a diagram. They might also confuse two different chemicals that sound similar or fail to connect a drawing of a molecule to the text describing its behavior. This gap between the way humans understand scientific evidence and the way computers search for it has become a significant bottleneck, slowing down the process of turning scientific ideas into new medicines.
To bridge this gap, a team of researchers at Texas A&M University and Panorama Global has developed a new system called DocuStore. Instead of treating drug discovery documents as simple blocks of text, this system is designed to understand the specific language of chemistry and biology. The researchers built a process that reads these documents and actively pulls out the most critical pieces of information: the actual shapes of molecules, the names of biological targets, and the exact numbers measuring how effective a drug is. Crucially, the system links all these different pieces together. It recognizes that a drawing of a molecule, a text label like "Compound 25a," and a table entry showing a specific measurement all refer to the same single entity. By creating these connections before a search even begins, the system ensures that when a scientist asks a question, the computer retrieves the right evidence, regardless of whether that evidence was originally written as a sentence, a number, or a picture.
The researchers tested this new approach against a standard search method using a carefully constructed set of 1,093 questions based on real medicinal chemistry presentations. These questions ranged from simple requests, such as "What is the strength of this specific compound?" to more complex inquiries requiring the comparison of data across multiple different documents. The results showed a clear and substantial improvement. When using the standard method, the computer provided the correct answer to only about 47 percent of the questions. With the new DocuStore system, the success rate jumped to 75 percent. Perhaps even more telling was the system's ability to find the actual evidence needed to answer the question. The standard method found the necessary supporting documents for only 42 percent of the questions, whereas DocuStore successfully located the complete evidence for 87 percent. This suggests that the system is not just guessing better; it is fundamentally finding the right information more often.
The power of this new system becomes most apparent when the question and the answer are presented in different formats. For example, if a researcher asks a question using a chemical drawing (a structure) but the answer is hidden in a text description of a compound with a specific code name, standard search tools often fail to make the connection. They see a picture and a word as unrelated. DocuStore, however, translates the picture into a digital code that it can match with the text. In tests where the question was posed as a chemical structure, the new system found the correct answer 80 percent of the time, while the standard method managed less than 20 percent. This ability to translate between different ways of representing the same scientific fact allows the system to retrieve evidence that would otherwise remain invisible to a computer.
Beyond just finding the right page, the system also improved how the computer formulated the final answer. When the system successfully retrieved the correct evidence, it used that information to generate a response that was accurate and supported by citations. The researchers found that the system was much less likely to guess or admit defeat when it didn't know the answer. Instead of giving up or providing a vague response, it was able to synthesize the specific data points it had found. This is particularly important in a field where a wrong answer could lead to wasted time and resources. The system also demonstrated an ability to handle follow-up questions in a conversation, remembering the context of a previous query to answer a new question about the same compound without needing the user to repeat the details.
The study also highlighted where the system still faces challenges, particularly when a question requires combining information from many different documents to perform a complex calculation or comparison. While the system became much better at finding the individual pieces of evidence across multiple reports, the final step of assembling those pieces into a complete, calculated answer remained difficult. This indicates that while the system has solved the problem of locating the right information, the next step involves teaching the computer how to perform the complex reasoning required to combine that information. Nevertheless, the findings demonstrate that incorporating specific chemical knowledge into the way computers search and understand documents can dramatically improve their performance. By treating scientific evidence as a connected web of structures, names, and numbers rather than just text, the researchers have created a tool that brings computer search capabilities closer to the way human scientists actually think and work.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.