SGHA: Evidence-Grounded Research Problem Discovery with Local Language Models
The paper introduces SGHA, a fully automated, local-language-model-based system that discovers evidence-grounded research problems by structuring scientific literature into a typed evidence graph to detect structural gaps, thereby enabling transparent and auditable problem formulation without relying on proprietary frontier models.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Science has long relied on the human mind to spot the next big question. A researcher reads dozens of papers, notices that a certain method keeps failing under specific conditions, or sees that a limitation is mentioned repeatedly without ever being solved, and then formulates a new problem to tackle it. This process of connecting dots across different studies is difficult, time-consuming, and deeply dependent on the researcher's own experience. In recent years, artificial intelligence has begun to assist with this work, capable of reading vast libraries of text and suggesting new ideas. However, many of these advanced systems rely on powerful, proprietary models that operate like black boxes. They generate suggestions based on hidden patterns in their training data, making it hard to verify where an idea truly came from or to audit the evidence behind it. This creates a risk where the AI might invent problems that sound plausible but have no real foundation in the scientific record, or it might hide the specific reasons why a problem was chosen.
A team of researchers has introduced a new approach called the Structural Gap Hypothesis Agent, or SGHA, which attempts to solve this by grounding the search for new research problems entirely in the text of the scientific papers themselves. Instead of asking a computer to imagine a new direction from scratch, this system treats the existing literature as a structured map. It reads a collection of papers, extracts specific facts about what methods work, what assumptions they rely on, and where they fail, and then builds a detailed network of these connections. The system then scans this network for "structural gaps"—patterns where the evidence suggests a problem exists but no one has yet written a formal research question to address it. For example, if many papers show that a technique works well when data is simple but fails when data is complex, and no paper has yet proposed a study to fix that specific failure, the system flags this as a gap. The goal is to produce research problems that are not just creative guesses, but are directly traceable to the evidence found in the literature, complete with a clear record of the source material and the specific uncertainties that remain.
The researchers tested this system across five different areas of machine learning, a field of computer science focused on teaching computers to learn from data. They fed the system a total of 1,250 scientific papers from these fields. The system successfully read and understood the text of 1,044 of these papers, pulling out thousands of specific facts and relationships to build its evidence map. From this massive collection of information, the system identified 39 potential gaps where the literature suggested a problem but had not yet defined a solution. It then subjected these 39 candidates to a strict verification process. In this stage, the system acted as its own critic, checking to ensure that each gap was truly supported by the papers and that the proposed problem was not just a repetition of work already done. Only those gaps that passed this rigorous check were allowed to move forward. The final result was a set of 15 formalized research projects, each presented as a structured problem statement that included the main variables, the assumptions being tested, the criteria for success, and a list of ambiguities that a human researcher would need to resolve before starting the work.
To see if this method was effective, the researchers compared the output of their system against other automated idea generators, including a well-known system called AI-Scientist. They used the same underlying language model for all comparisons to ensure a fair test, focusing on the quality of the research problem formulation rather than the ability to write code or run experiments. The results showed that the new system produced research problems that were significantly better at connecting back to the source material. The problems generated by SGHA were more specific about where the evidence came from, clearer about the assumptions being made, and more honest about what parts of the problem were still undefined. While the other systems produced ideas that were often well-written and technically detailed, they tended to lack the deep connection to the specific evidence in the papers and often missed the opportunity to explicitly state the limitations or uncertainties of the proposed work. The new system's output was more like a detailed blueprint that a researcher could pick up and immediately understand the foundation of, whereas the others were more like general proposals.
The researchers also explored how the system could be tailored to a specific scientist's interests. By feeding the system a profile of a researcher's past work, they found it could prioritize gaps that were most relevant to that person's history, effectively acting as a personalized research assistant. They also tested a mode that allowed the system to explore a wider range of possibilities, generating many more potential directions, though these were less polished and required more human refinement. The study suggests that the size of the library of papers matters; with too few papers, the system could not find enough evidence to form a solid problem, but with a rich collection, it could identify complex patterns. However, the system did not simply output a fixed number of ideas; it stopped when the evidence was not strong enough to support a new problem, refusing to force a solution where none was warranted.
This work highlights a shift in how artificial intelligence might assist science. Rather than replacing human intuition with a black box that generates endless ideas, this approach uses AI to organize the vast amount of existing knowledge into a clear, auditable structure. It demonstrates that by carefully mapping the relationships between what is known, what is assumed, and what is missing, a computer can help identify the most promising questions to ask next. The system does not claim to have solved these problems or to have found the next great discovery on its own. Instead, it offers a grounded starting point, turning the scattered signals of the scientific literature into a clear, inspectable set of research directions that human scientists can then take, refine, and pursue.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.