When More Modalities Hurt: Modality Dropout for Heavy-Duty Vehicle Engine Diagnostics
This paper demonstrates that applying modality dropout during training significantly improves heavy-duty vehicle engine diagnostics by forcing models to effectively fuse unstructured service complaints, sparse sensor telemetry, and Diagnostic Trouble Codes, achieving a 68.8% accuracy that outperforms both text-only baselines and traditional machine learning methods.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
When a massive truck breaks down, the story of the failure is told in three different languages at once. A technician writes a note in a service log, often describing what they see or hear in a mix of languages like English, German, or Polish. At the same time, the truck's computer records a stream of numbers from hundreds of sensors, measuring things like pressure and temperature, though many of these readings are missing because the sensors were not active at that moment. Finally, the computer itself generates a structured code, a specific fault indicator that names the exact part that crossed a safety threshold. For decades, mechanics and engineers have looked at these three sources of information separately, treating the written note, the sensor numbers, and the fault code as isolated clues. The question driving new research is whether combining these three distinct streams of data into a single picture would help a computer diagnose the problem more accurately, or if the written note alone is already so rich in detail that the other data adds nothing but confusion.
A team of researchers at Halmstad University in Sweden set out to test this idea using real-world data from a major heavy-duty truck manufacturer. They gathered a dataset of nearly nine hundred specific engine failures, where each case included the technician's written complaint, the available sensor readings, and the fault codes generated by the vehicle. Their goal was to build a computer system that could look at these three inputs and correctly identify which of five major engine systems was broken: the mechanical parts, the fuel system, the cooling system, the intake and exhaust, or the electronics. They found that simply throwing all three types of data together did not work well. In fact, when they fed the computer the text, the sensors, and the codes all at once without special training, the system performed only slightly better than if it had just read the technician's notes. The extra data seemed to drown out the clear signal from the text rather than helping it.
The breakthrough came when the researchers changed how they taught the computer to learn. Instead of letting the system see all three data streams at every step, they introduced a technique where they randomly turned off entire streams of data during the training process. Imagine a student learning to solve a puzzle who is occasionally forced to solve it using only half the pieces; this forces the student to learn how to use every piece effectively rather than relying on just the easiest ones. By randomly disabling the sensor data or the fault codes during training, the computer was forced to learn how to extract useful information from the text even when the other data was missing, and to understand the fault codes even when the text was vague. This method, known as modality dropout, prevented the system from ignoring the weaker data sources.
The results showed that this training strategy made a significant difference. The best-performing system combined the technician's written notes with the fault codes, but only after being trained with this random dropout method. This approach achieved an accuracy of nearly sixty-nine percent, a clear improvement over the sixty-five percent accuracy reached by using the text alone. The researchers discovered that different types of engine problems required different kinds of evidence. For issues with the air intake and exhaust, the raw sensor numbers were the most powerful clue, correctly identifying the problem ninety-three percent of the time, far outperforming the written descriptions. However, for problems with the fuel system, the written notes alone were almost useless, getting the answer right only fifteen percent of the time. In these difficult cases, combining the notes with the fault codes and using the dropout training method nearly tripled the success rate to thirty-eight percent.
The study concluded that while combining data sources is powerful, it requires a careful approach to ensure the computer learns to value every piece of information. Simply merging text, sensor numbers, and fault codes does not automatically lead to better answers; in fact, without the right training, the extra data can be a hindrance. The most effective strategy was to teach the system to be flexible, forcing it to find the solution using whatever combination of clues was available. This approach proved that the written notes, the structured fault codes, and the physical sensor readings each hold unique value, but their true power is unlocked only when the system is trained to respect and utilize each one, especially when the data is incomplete or messy. This work represents the first time these three specific types of industrial data have been fused together to diagnose heavy-duty vehicle engines, offering a new path for making maintenance more reliable and efficient.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.