Better Call Graphs: A New Dataset of Function Call Graphs for Malware Classification
This paper introduces "Better Call Graphs" (BCG), a comprehensive and publicly available dataset of unique, large-scale function call graphs extracted from recent Android applications, designed to overcome the limitations of existing outdated and redundant datasets by enabling more reliable and diverse malware classification.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the digital world, mobile applications are not just collections of code; they are intricate networks of instructions that tell a phone what to do. To understand how an app behaves, security researchers often look at its "function call graph." Imagine this as a map of a city where every building is a specific task the app can perform, and the roads connecting them show which tasks trigger others. This map reveals the app's true structure and behavior, which is far more reliable for spotting trouble than simply reading the app's name or looking at a few surface-level details. This approach is crucial because bad actors who create malicious software, or malware, often try to hide their work by making tiny, cosmetic changes to the code. These changes can fool simple scanners that look for specific text patterns, but they rarely alter the underlying map of how the app actually functions. For years, the field of mobile security has relied on these maps to catch threats, but the quality of the maps themselves has been a major bottleneck.
For a long time, researchers trying to build better detection systems have been working with outdated or flawed maps. Many existing collections of these app maps were built from software that is now decades old, reflecting a time when mobile apps were much simpler. Furthermore, these older collections were often filled with copies of the same app, disguised with slightly different names but sharing the exact same internal structure. This created a false sense of security; machine learning models trained on these old, repetitive maps could score very high marks in tests, not because they were smart, but because they had simply memorized the duplicates. When faced with a new, modern app, these models often failed completely. The rapid evolution of the mobile ecosystem means that the threats today are structurally different from those of the past, and the tools used to study them needed a serious update.
To solve this problem, a team of researchers at the University at Buffalo has created a new, comprehensive collection of app maps called "Better Call Graphs." This dataset is not just a larger version of what came before; it is a fundamentally different resource built from scratch using recent software. The team gathered over 125,000 application files from major public repositories, filtering them down to a final set of roughly 10,000 unique examples. They were strict about quality: they discarded any app that was too small to contain complex behavior, removed any software compiled before 2017 to ensure modern relevance, and rigorously checked for duplicates. They used a multi-engine verification system to confirm which apps were malicious, ensuring that the labels were accurate and reliable. The result is a dataset that captures the true complexity of modern mobile software, containing both harmless apps and various types of malware, including trojans, adware, and ransomware.
When the researchers tested standard detection methods on this new dataset, the results were stark and revealing. On older, familiar datasets, the best machine learning models achieved accuracy rates near 90 percent, making the task of spotting malware seem almost solved. However, when these same models were applied to the new "Better Call Graphs" dataset, their performance plummeted. The accuracy dropped to around 77 percent, and a more critical measure of success, known as the macro-F1 score, fell from nearly 90 percent to just 18 percent. This dramatic decline was not a failure of the models, but rather a proof that the old datasets had been misleading. The models had been overfitting to the repetitive patterns of the past, and when faced with the genuine complexity and variety of modern apps, they struggled to generalize. The new dataset exposed a significant gap between current technology and the reality of modern mobile threats.
The study also explored how the difficulty of detection changes over time. When the researchers trained their models on older apps and tested them on newer ones, the performance dropped even further, confirming that the landscape of mobile malware is shifting rapidly. This phenomenon, known as concept drift, means that a model trained on data from five years ago is likely to be ineffective today. The new dataset also included detailed information about the apps themselves, such as their size, the permissions they request, and the specific libraries they use, allowing for a more holistic view of the software. Despite these rich details, even the most advanced graph-based models found it difficult to distinguish between different families of malware within the new dataset, suggesting that the structural similarities between different types of modern threats are much stronger than previously thought.
Ultimately, this work serves as a necessary reality check for the field of mobile security. By providing a clean, modern, and non-redundant collection of app maps, the researchers have established a new standard for how detection systems should be evaluated. The findings suggest that the high success rates reported in previous studies were often an illusion created by outdated and repetitive data. The path forward requires developing new techniques that can handle the genuine complexity of modern applications, rather than relying on methods that only work on the simplified, historical versions of the problem. The dataset is now publicly available, offering a rigorous testbed for future research and ensuring that the next generation of security tools is built on a foundation of truth rather than repetition.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.