Assessing LLMs' mathematical abilities requires understanding the various mechanisms of mathematical creativity
The paper argues that assessing LLMs' mathematical abilities requires moving beyond aggregate benchmarks to a nuanced taxonomy of distinct, non-substitutable creative mechanisms, suggesting that while current models excel at recombining existing knowledge, they may fundamentally lack the capacity for other forms of mathematical invention that are becoming increasingly valuable as automated proof generation improves.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Mathematics has long been viewed as the ultimate test of human logic, a realm where answers are either right or wrong, and where the path to truth is paved with rigorous proof. For centuries, the field relied on a simple economy: human intuition would spot a pattern or a possibility, and then the hard work of verification would follow to confirm it. Today, artificial intelligence has begun to master the second half of this equation. Modern computer systems can now generate and check mathematical proofs at a speed and scale that far outstrips human capability, turning the once-scarce resource of verification into an overwhelming abundance. This shift forces a new, more difficult question: if machines can prove things so easily, where does the true value of mathematics lie? The answer lies not in the proof itself, but in the spark that starts the whole process—the moment a mathematician invents a new idea, a new concept, or a new way of seeing the world.
A recent paper by Silvère Gangloff investigates exactly this spark, asking whether artificial intelligence can truly invent mathematical ideas or if it is merely rearranging old ones. The author argues that we have been asking the wrong question. Instead of treating "mathematical creativity" as a single skill that a computer either has or lacks, the paper suggests it is actually a collection of distinct, unrelated mechanisms. Some of these mechanisms involve looking at how mathematicians work and turning those habits into formal rules. Others involve taking a structure discovered in the physical world, like the way heat flows, and repurposing it to solve abstract puzzles. There is also the work of solving a specific, stubborn problem by inventing a new tool just to crack it, and the rare, high-level act of connecting two fields of math that have never spoken to each other before. The paper proposes that these are not just different flavors of the same thing; they are fundamentally different types of thinking that require different kinds of machinery.
The central finding of the research is that current artificial intelligence systems are exceptionally good at some of these modes but likely incapable of others, no matter how much more powerful they become. The systems we have today operate by recombining existing pieces. They are like master librarians who can instantly find and stitch together any two facts they have read before. This makes them brilliant at finding a specific solution to a known type of problem or at checking if a proof follows the rules. However, the paper suggests they hit a hard wall when asked to invent a completely new type of object or a new way of thinking that has no precedent in their training data. For instance, a computer can search through millions of existing mathematical structures to find one that fits a constraint, but it cannot look at the practice of human calculation and decide, on its own, that this practice itself should become a new mathematical object to study. This is a difference in kind, not just speed; the machine is not just slower at inventing new primitives, it is structurally unable to do so because it lacks the mechanism to select what is worth inventing in the first place.
The author supports this view by looking at how mathematics has actually advanced throughout history. In one famous case, a mathematician named Alan Turing did not just solve a problem; he looked at the act of a human following a rulebook and decided to make that act the subject of a new theory. In another, a physicist named Kolmogorov took a concept from thermodynamics, which deals with heat and energy, and applied it to pure mathematics to solve a problem about abstract shapes. These were not searches for a missing piece in a known puzzle; they were acts of creating the puzzle itself. The paper argues that current artificial intelligence, which relies on searching through a fixed library of known moves, cannot replicate this. It can find the missing piece if the shape of the piece is already defined, but it cannot imagine a new shape that has never existed.
This distinction has profound implications for how we should judge the future of artificial intelligence in science. If we continue to measure AI success by how many proofs it can generate or how well it solves standard test problems, we are measuring the wrong thing. We are rewarding the machine for doing the work it is already good at, while ignoring the work that actually drives human progress. The paper warns that as proof generation becomes cheap and automatic, the true value of mathematics will migrate toward these harder modes of invention: the reflexive act of formalizing new practices, the analogical leap of importing ideas from other fields, and the deep, goal-driven creation of entirely new concepts. If we do not change how we evaluate these systems, we risk creating a flood of correct but uninteresting results, while the genuine, transformative leaps in understanding remain out of reach.
The research does not claim that artificial intelligence will never be able to perform these creative acts. It suggests that for them to happen, the underlying technology would need to change, perhaps by developing systems that can interact with the world to form grounded understandings, rather than just predicting the next word in a sentence. Until then, the paper concludes, we must be careful not to confuse the ability to recombine old ideas with the ability to create new ones. The future of mathematical discovery may depend less on how fast our computers can calculate, and more on whether we can build machines that know what is worth inventing.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.