Malformer: A Multi-Modal Malware Detector Using Transformers
The paper introduces Malformer, a quadrimodal transformer-based detector that fuses text, image, graph, and audio representations of Windows executables to achieve 98.3% accuracy, significantly outperforming traditional unimodal and bimodal malware detection systems.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the digital world, security teams face a constant, shifting battle against malicious software. For decades, the primary defense has been to look for known fingerprints: specific patterns of code or unique digital signatures that identify a virus. This approach works well against old threats, but it fails when attackers create new, unseen programs or slightly alter existing ones to hide their tracks. To stay ahead, researchers have turned to machine learning, teaching computers to recognize the subtle, underlying structures of bad software rather than just matching known patterns. Early attempts taught computers to read the raw code or to look at the file as a simple image, but each method had blind spots. A single way of looking at a file might miss critical clues, much like trying to identify a person by only seeing their shadow or only hearing their voice.
A team of researchers from Tennessee Tech University, the University of Nebraska at Omaha, and North Carolina A&T State University has developed a new system called Malformer to overcome these limitations. Instead of relying on just one view of a computer file, their system looks at the same file in four completely different ways at once. They treat the file as a text document, an image, a map of connections, and a sound wave. By combining these four perspectives, the system builds a much richer understanding of what the file is trying to do. The researchers tested this approach on a massive collection of over 200,000 computer files and found that looking at the problem through all four lenses simultaneously allowed the system to spot threats with significantly higher accuracy than methods using only one or two views.
The core idea behind Malformer is that different ways of representing a file reveal different kinds of information. When a computer file is converted into text, it shows the specific instructions the program gives to the processor. When that same file is turned into an image, it reveals patterns in the arrangement of its data, similar to how a fingerprint shows ridges and valleys. When converted into a graph, it maps out how different parts of the program talk to each other, showing the flow of control. Finally, when treated as an audio signal, the sequence of bytes in the file creates a sound wave that carries rhythmic patterns. The researchers discovered that relying on just one of these views often leaves gaps. For instance, if an attacker obfuscates the text code to hide it, the image or sound view might still show the malicious structure. By fusing all four views, the system creates a safety net where one view can compensate for the weaknesses of another.
To build this system, the researchers used advanced tools known as transformers, which are powerful computer models capable of understanding complex relationships in data. They adapted these models to handle the four different types of data. For the text and the connection map, they used models designed to understand language and sequences. For the image, they used a model that typically analyzes photographs but modified it to see the byte patterns of a file. For the sound, they used a model trained on human speech to listen to the rhythm of the file's data. The challenge was not just in creating these four separate views, but in teaching the system how to weigh them against each other. The researchers developed a method that automatically adjusts how much attention the system pays to each view during training. If one view is learning faster than the others, the system temporarily reduces its influence to ensure the slower views catch up, preventing the whole system from becoming biased toward just one type of information.
The results of this approach were striking. When tested on a dataset of 201,549 files, the Malformer system correctly identified malicious software 98.3% of the time. This performance was notably better than systems that looked at only one type of data or even those that combined just two. The researchers found that the combination of these four views provided a level of detail that no single view could achieve alone. Even more importantly, the system proved to be resilient. In real-world scenarios, some parts of a file might be damaged or impossible to read, causing one of the four views to fail. The researchers tested what would happen if they removed the text view or the connection map view, and the system still performed very well, only dropping slightly in accuracy. This suggests that the system does not rely on a single point of failure but rather on the collective strength of all four perspectives.
The study also highlighted the unique value of each perspective. While the image view was the most consistent and reliable, the connection map view provided the strongest individual signal when it was available. The text view, which represents the assembly code, was particularly useful when the connection map failed to extract properly. The sound view, which is a less common approach in this field, also proved effective, capturing patterns that the other views missed. By bringing these diverse methods together, the researchers created a detector that is not only more accurate but also more robust against the tricks attackers use to hide their work. The work suggests that the future of malware detection lies not in finding a single perfect way to analyze a file, but in integrating multiple, complementary ways of seeing the same threat. As attackers continue to evolve their methods, having a system that can look at a problem from four different angles simultaneously offers a powerful and generalized defense for the digital infrastructure we rely on every day.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.