AnoMLP-TCP: An Anomaly Aware Multilayer Perceptron for Test Case Prioritization in Continuous Integration
The paper introduces AnoMLP-TCP, a lightweight anomaly-aware multilayer perceptron for test case prioritization in continuous integration that achieves performance comparable to the more complex AnoLSTM-TCP baseline while significantly reducing model parameters, training time, and computational cost.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the world of modern software development, teams build and update their programs constantly, often multiple times a day. To ensure these frequent changes do not break the system, engineers run a massive battery of automated checks known as regression tests. These tests act as a safety net, verifying that the software still works as intended after every new piece of code is added. However, as software grows more complex, these test suites become enormous. Running every single test after every tiny change takes too long, creating a bottleneck that slows down the entire development process. Developers wait hours for feedback, and by the time a problem is found, the cause is often harder to trace. To solve this, researchers use a strategy called test case prioritization. Instead of running tests in a random order or by their original creation date, this method rearranges the list so that the tests most likely to fail are run first. The goal is simple: find the bugs as quickly as possible so the team can fix them before the day is over.
For years, the most advanced methods for solving this sorting problem have relied on complex artificial intelligence models that mimic the way human brains process sequences of information over time. These models, known as recurrent neural networks, are powerful but heavy, requiring significant computing power and time to train. A researcher at Abubakar Tafawa Balewa University in Nigeria asked a straightforward question: is all that complexity actually necessary? They wondered if a simpler, more direct type of artificial intelligence could achieve the same results without the heavy computational cost. To find out, they developed a new approach called AnoMLP-TCP, which replaces the complex, time-traveling model with a streamlined, feedforward network that looks at the data all at once.
The researcher tested their idea using real-world data from three different industrial projects at ABB Robotics in Norway. These projects involved thousands of automated tests running over hundreds of cycles, providing a rich history of which tests passed, which failed, and how long they took. The team fed this historical data into their new model, along with a special signal that flags unusual patterns in the test history. This signal, derived from a technique called anomaly detection, helps the system spot when a test is behaving strangely, which often precedes a failure. The model then learned to assign a priority score to each test, predicting which ones were most likely to break the software if run next.
When the researcher compared their new, lightweight model against the heavy, complex standard it was designed to replace, the results were surprising. The simpler model performed just as well as the complex one. In terms of how quickly it could find faults, the two models were identical. On one dataset, both found faults with a success rate of nearly 96 percent; on another, they both reached about 89 percent. The researcher found that the complex model did not offer any advantage in sorting the tests effectively. The only difference was in the cost of running them. The new, simpler model required roughly 76 percent fewer internal settings, known as parameters, to function. Because it was so much smaller, it trained significantly faster, taking less than half the time per cycle to learn from the data compared to its complex counterpart.
The study also looked at what happens when a team cannot run all their tests due to time limits, a common scenario in fast-paced development. Even when the researcher simulated running only 50 percent, 70 percent, or 90 percent of the tests, the new model continued to match the performance of the complex one perfectly. It identified the same critical tests to run first, ensuring that the most important faults were caught regardless of how much time was available. The researcher noted that this equivalence was not a fluke but a consistent result across all three industrial datasets they examined. They also discovered that the complex model they were trying to improve upon had been suffering from a technical glitch in its design that limited its ability to distinguish between different test scores, a problem their simpler model avoided entirely by using a different mathematical structure.
This work suggests that for the specific task of prioritizing software tests in continuous integration, the industry may not need the most sophisticated, heavy-duty artificial intelligence models available. The researcher demonstrated that a much simpler, more efficient architecture can do the job just as effectively. By reducing the computational burden, this approach allows teams to update their prioritization models more frequently and with less energy, making the process of software testing more agile. The findings indicate that in this specific corner of software engineering, the path to better efficiency does not always require building bigger, more complex machines, but sometimes just finding a clearer, more direct way to look at the data.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.