An Investigation of the NeurIPS and ICML 2025 Position Tracks

In the world of artificial intelligence research, conferences act as the great gatekeepers of progress. When scientists want to share a new idea, they submit it to these gatherings, where a panel of peers decides if the work is good enough to be published and discussed. Among the most prestigious of these gatherings are NeurIPS and ICML, which set the tone for what the entire field considers important. Within these massive conferences, there is a special section called the "Position Paper Track." Unlike standard research papers that present new data or a finished tool, position papers are meant to be visionary. They are supposed to argue for a new direction, challenge the community to think differently, or propose a fresh way of solving problems. The goal is not just to fix a small error in an existing method, but to shift the entire agenda of the field, much like a captain steering a ship toward a new horizon rather than just polishing the deck.

However, a recent investigation into the 2025 submissions for these tracks suggests that the compass might be pointing in a narrower direction than intended. Researchers Fan Yang, Wenkai Li, and Jun Liu decided to audit the publicly available pool of papers from these two major conferences to see what kinds of arguments were actually being made. They looked at 191 papers that had been reviewed and found a striking pattern: the vast majority were not setting new courses. Instead, they were focused on fixing what already existed. Roughly three-quarters of the papers were "reformist critiques," meaning they argued that current benchmarks, evaluation methods, or datasets were flawed and needed adjustment. While these critiques were often rigorous and valuable, they represented a different kind of work than the bold, agenda-setting vision the track was designed to host. The authors found that papers proposing entirely new directions were rare, and the papers that did propose them did not necessarily receive higher scores from reviewers.

The researchers compared this current landscape with a "reference class" of famous papers from the past decade that successfully shifted the field's focus, such as the work that introduced the Transformer architecture or the paper that defined concrete safety problems in AI. These historical turning points shared a common trait: they did not just argue about what was wrong; they gave the community something new to build on. They provided a new measurement protocol, a toy implementation, a dataset card, or a falsifiable experiment that others could immediately use to test the new idea. In contrast, the 2025 reviewed pool was dominated by papers that critiqued existing tools without offering a new operational framework to replace them. The study suggests that the conference review process has inadvertently become a filter that favors safe, incremental corrections over risky, visionary proposals. This is not because reviewers are rejecting ambitious ideas out of hand, but because the system rewards work that is easy to verify in the short term, leaving the harder, more speculative work of defining the future largely unexplored.

To understand why this imbalance exists, the authors proposed several reasons rooted in how human beings and institutions operate. First, it is easier to prove that a current method is broken than to prove that a new, untried direction is the right one. A critique of an existing benchmark can be tested with a simple experiment, whereas a proposal for a new research path requires a leap of faith that is harder to defend in a written review. Second, the culture of machine learning heavily rewards empirical numbers and rigorous testing, which naturally favors papers that can produce immediate data over those that offer a new perspective. Third, career incentives play a role; a junior researcher might feel that proposing a radical new direction is too risky for their job prospects, whereas critiquing an existing practice is a safe, citable contribution. The combination of these pressures creates a feedback loop where the visible pool of papers teaches future authors that "position papers" are simply critiques of the status quo, rather than blueprints for the future.

The authors do not argue that the reformist critiques are bad or that they should be removed. In fact, they acknowledge that these papers are often high-quality and necessary. The problem is one of balance. If a venue created to set the agenda becomes dominated by papers that only adjust the current agenda, the field may lose its ability to pivot toward truly new frontiers. The study concludes that the solution does not require a complete overhaul of the review process, but rather a few targeted nudges. The authors suggest that conference organizers could explicitly ask authors to state whether their paper is a critique of the present or a proposal for the future. They also recommend requiring a sentence that explains how a new idea could be proven wrong, forcing authors to make their claims more concrete. Finally, they propose that organizers should actively solicit papers that bundle a new perspective with a tangible artifact, such as a new dataset or a simple experimental program, to make the vision operational. By making these small changes, the community can ensure that the position paper track fulfills its original promise: not just to fix what is broken, but to show us where to go next.