A Benchmark Dataset for Graph Regression with Homogeneous and Multi-Relational Variants
This contribution introduces RelSC, a novel benchmark dataset for graph regression derived from program graphs with runtime labels and offered in both homogeneous and multi-relational variants to evaluate how structural representation decisions influence model performance.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to teach a robot to estimate how long a software program takes to execute. To do this, you must show the robot an image of the code. But not just any image—a special map that represents how the different parts of the code communicate with each other.
This article introduces a new, massive "training hall" (a dataset) called RelSC to help researchers develop better robots (AI models) for this specific task. Here is a breakdown of what they did, using simple analogies.
The Problem: The Robot's Diet Is Too Boring
Currently, most AI models that analyze graphs (maps of connections) are fed a very limited diet. They mainly eat molecules (like chemical compounds used to create medicines) or citation networks (like a map showing who cited whom in scientific papers).
The authors argue this is like a chef who only knows how to cook with apples. They want to teach the AI to cook with everything, including software code. Yet, there was no good "cookbook" (dataset) for software performance.
The Solution: A New "Code Training Hall" (RelSC)
The authors created RelSC, a massive collection of Java programs paired with their actual "execution times" (how long they took to run). Imagine this as a library where every book (code) comes with a stopwatch.
They built this library in two different "flavors" to test how the AI learns:
RelSC-H (The homogeneous version):
- The Analogy: Imagine a city map where every road is just a "road." You can see the roads, but you don't know if a road is a highway, a dirt path, or a bike lane. It is all just a "connection."
- In the Article: This version transforms the code into a graph where all connections look the same, but the "buildings" (nodes) contain rich details about what they are (e.g., "this is a mathematical operation," "this is a variable").
RelSC-M (The multi-relational version):
- The Analogy: Now imagine the same city map, but the roads are color-coded and labeled. You have highways (data flowing from one variable to another), traffic lights (if/else decisions), and one-way streets (loops).
- In the Article: This version preserves the specific "types" of connections. It tells the AI: "This line connects a variable to a mathematical operation" or "This line connects a condition to a loop." It is a much more detailed, complex map.
How They Created the Maps
To transform code into these maps, they used three standard computer science tools, like layers of a cake:
- AST (The Skeleton): The basic structure of the code (like the frame of a house).
- CFG (The Traffic Flow): How the program moves (like traffic lights and turn signals).
- DFG (The Water Pipes): How data moves and changes (like water flowing through pipes).
They blended these three into a super-detailed map of code behavior.
The Experiment: Who Learned Best?
The authors placed various AI models (Graph Neural Networks) into this training hall to see how well they could predict execution time.
- The Results:
- The AI models that used the graph maps (RelSC) were generally better at estimating time than models that simply read the code as text or simple trees.
- Surprising Insight: Although RelSC-M (the detailed, multi-lane highway map) contained more information, the models sometimes performed better with RelSC-H (the simpler, single-lane map).
- The Takeaway: This suggests that too many details, or the wrong kind of details, can sometimes confuse the AI. It is like giving a driver a map where every single pothole is marked; sometimes a simpler map is easier to navigate.
Why This Matters
The article claims this dataset is a "challenging and versatile benchmark." It forces AI researchers to stop testing only on molecules and start testing on real software structures.
In short: The authors built a new, diverse training ground for AI to learn how to predict software speed. They showed that while detailed maps of code are powerful, the way we draw these maps is just as important as the information within them. They are now making this "training hall" available to everyone so others can try to build better robots.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.