CEDAR-GRPO: Process-Aware Reinforcement Learning for General Abductive Reasoning in LLMs

In the vast landscape of artificial intelligence, researchers are constantly trying to teach machines how to think like humans, not just how to predict the next word in a sentence. One of the most human ways of thinking is called abductive reasoning. This is the mental process of looking at a set of clues or observations and working backward to find the most likely explanation for them. It is how a doctor figures out what is wrong with a patient based on a few symptoms, how a detective reconstructs a crime from scattered evidence, or how an engineer diagnoses a machine that has suddenly stopped working. For a long time, scientists have struggled to get large language models to do this well. While these models are excellent at recalling facts or following strict logical rules, they often stumble when asked to infer hidden causes from incomplete information. They tend to guess the right answer for the wrong reasons, or they make up facts that sound plausible but do not actually fit the evidence provided.

A team of researchers at Sharif University of Technology and the University of Tehran set out to solve this problem. They wanted to know if they could train these AI models to become better at abductive reasoning in a way that would help them solve new, unseen problems, rather than just memorizing the specific puzzles they were trained on. They developed a new training method called CEDAR-GRPO. Instead of simply rewarding the model for getting the final answer right, they designed a system that also rewards the model for how it got there. The system checks two specific things about the model's thought process: first, whether the model actually looked at and explained every single detail in the evidence provided, and second, whether the model moved its reasoning in the correct direction, starting from the observations and building toward a conclusion, rather than starting with a conclusion and trying to force the evidence to fit it.

To test this, the researchers took four different open-source language models and trained them on a carefully mixed set of tasks. These tasks included everything from choosing the best explanation for a short story to writing code that explains a pattern in data. Crucially, they did not just show the models the answers; they used a reinforcement learning technique that acted like a coach, giving the models feedback on their reasoning steps. The models learned that a correct answer was not enough if they had ignored a key detail or if their logic was backwards. After this training, the researchers tested the models on eleven completely different tasks that they had never seen before. These new tasks ranged from diagnosing medical conditions and debugging computer code to solving complex mysteries and understanding long, complicated stories.

The results showed a clear improvement across the board. The models trained with the new method outperformed both their original versions and models that were trained only to get the final answer correct. On average, the new method improved performance by 7.4 percentage points over the original models and by 2.7 points over the models trained only for correctness. In some specific cases, the improvement was as high as 30.8 points. More importantly, the researchers analyzed the actual text the models wrote while they were thinking. They found that the trained models were behaving more like careful investigators. They were more likely to explore different possibilities before settling on an answer, to explicitly rule out wrong ideas, and to admit when they were uncertain. They also showed a much stronger habit of linking their conclusions directly back to the specific facts they were given, rather than relying on vague assumptions.

The study also ruled out several alternative explanations for this success. The researchers proved that the improvement did not come simply from the models seeing more examples or from a general boost in reasoning ability. When they trained the models on a similar amount of general reasoning data that did not focus on abductive tasks, the models did not show the same level of improvement on the specific tests. This suggests that the specific way the models were rewarded for their reasoning process was the key factor. The findings indicate that abductive reasoning can be strengthened as a general skill that transfers across different fields, rather than being a narrow trick that only works on specific types of puzzles. By teaching the models to respect the evidence and follow a logical path from observation to explanation, the researchers have taken a significant step toward creating artificial intelligence that can truly understand and explain the world around it.