Beyond Journals: Rethinking Research Evaluation in Hungarian Computer Science
This study argues that Hungary's journal-centric research evaluation system misaligns with international computer science norms by undervaluing top-tier conference publications, leading to incentive distortions and the migration of successful researchers abroad, and recommends equating CORE A* and A conference papers with D1 and Q1 journals in national assessments.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The Big Picture: The "Wrong Scoreboard"
Imagine a sports league where the rules say that only goals scored in the slow, 90-minute league matches count toward a player's career stats. But in this specific sport (Computer Science), the most exciting, high-stakes, and important games are actually played in fast-paced, 3-day tournaments.
In Hungary, the "referees" (university evaluators and funding bodies) have been keeping score based on the old league matches (academic journals). However, the best computer scientists in the world are playing in the tournaments (conferences). Because the Hungarian scoreboard doesn't count tournament wins, it's creating a confusing situation where researchers have to choose between playing the game the world plays or playing the game Hungary demands.
The Core Problem: Journals vs. Conferences
For a long time, science was like a library. You wrote a long, detailed book (a journal article), waited a year for it to be edited, and then put it on the shelf. This was the gold standard.
But Computer Science is different. It's more like a live music festival.
- Speed: The music (new ideas) changes so fast that waiting a year to publish a "book" means the song is already old news.
- The Venue: The most important discoveries happen at big festivals (top conferences like NeurIPS or ICSE), not in the quiet library.
- The Ranking: There is a global "Festival Guide" called CORE. It ranks these festivals from A* (the biggest, most exclusive festivals) down to C (smaller local jams).
- A* is like playing at Wembley Stadium.
- A is like playing at a major arena.
- B and C are smaller venues.
The Hungarian Issue: Hungary's evaluation system is still obsessed with the "Library Books" (Journals). It treats a paper in a top-tier conference (A*) as if it were a minor publication, or ignores it entirely. This forces Hungarian researchers to write "books" just to satisfy local rules, even if the world is listening to the "live music."
What the Data Shows
The authors looked at the data and found some interesting patterns:
The "Theoretical" vs. "Applied" Split:
- Theoretical Computer Science (the math and logic behind the code) has been playing in these big festivals since the 1970s. Hungarian researchers there adapted quickly.
- Applied Computer Science (building actual software and AI) started playing in these festivals later (1990s). Hungarian researchers here are still struggling to get used to the new rules.
The "Brain Drain" Effect:
- The study found that the most successful Hungarian researchers (the ones who play in the A* festivals) are increasingly leaving Hungary.
- The Analogy: Imagine a local soccer team that refuses to let its star players play in the World Cup because "we only count local league games." Eventually, the star players leave to join teams in other countries that do count World Cup games.
- The data shows that many top Hungarian researchers are now working abroad or in international industry, while those who stay often have to split their time between writing "books" for Hungary and "playing festivals" for the world.
The "Double Bookkeeping" Nightmare:
- Currently, a researcher might publish a breakthrough at a top conference (A*). To get a promotion or funding in Hungary, they often have to take that same work, rewrite it, and publish it again in a journal.
- The Metaphor: It's like a chef winning a gold medal at a cooking competition, but the local government says, "That doesn't count. You must write a 50-page recipe book about the dish to get a salary." It's wasted time and energy.
Why This Matters for Everyone
The authors argue that this isn't just a problem for computer scientists; it affects the whole country.
- The "Operating System" Analogy: Computer Science is the operating system for modern research. Whether you are studying biology, medicine, or engineering, you need computer experts to run the simulations and analyze the data.
- If Hungary's best computer scientists are forced to stop playing the "global game" to satisfy local rules, the whole country loses out on the latest tools and techniques. It's like a country trying to build a modern city but refusing to use the latest construction technology because "we only use bricks from 1950."
The Proposed Solution
The authors suggest a simple rule change to fix the scoreboard:
- Treat an A Conference Paper like a D1 Journal Paper.* (The highest tier of journals).
- Treat an A Conference Paper like a Q1 Journal Paper. (The top 25% of journals).
They argue that the CORE ranking is already trusted globally (used by China, Australia, and others) and is reliable. Hungary doesn't need to invent its own list; it just needs to recognize the existing one.
The Bottom Line
The paper concludes that Hungary is taking small steps in the right direction, but it needs to go further. By aligning its rules with the rest of the world, Hungary can stop forcing researchers to play two different games. This will help keep young talent in the country, ensure Hungarian science stays competitive, and stop the "waste" of rewriting papers just to fit a local mold.
In short: Stop judging a fish by its ability to climb a tree. If Computer Science lives in conferences, let's start counting conferences.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.