AnoMLP-TCP: A Lightweight Feedforward Neural Network Approach for Anomaly-Aware Test Case Prioritization in Continuous Integration

In the world of modern software development, programs are rarely built once and left alone. Instead, they evolve constantly, with new features added and old ones tweaked in a rapid cycle known as continuous integration. Every time a change is made, the entire system must be tested to ensure nothing has broken. This process, called regression testing, is essential for quality, but it has become a bottleneck. As software grows, the list of tests needed to check it becomes enormous. Running every single test after every small change takes too long, delaying feedback to the engineers who need it. To solve this, researchers use a strategy called test case prioritization. This approach does not remove tests; instead, it rearranges them, placing the tests most likely to find a problem at the very front of the line. The goal is simple: find the faults as early as possible so they can be fixed before the day is over.

For years, the most advanced methods for doing this prioritization have relied on complex, deep learning models that mimic the way the human brain processes sequences of information over time. These models, known as recurrent neural networks, are powerful but heavy, requiring significant computing power and time to train. A team of researchers at Abubakar Tafawa Balewa University in Nigeria asked a fundamental question: is all that complexity actually necessary? They wondered if a much simpler type of artificial intelligence, one that looks at data in a single pass rather than remembering a long history, could achieve the same results. To find out, they developed a new, lightweight system called AnoMLP-TCP and put it to the test against the more complicated, established models using real-world data from ABB Robotics in Norway.

The researchers worked with three distinct sets of data from industrial robots, containing thousands of test runs and millions of recorded outcomes. These datasets included information such as how long a test took to run, when it was last executed, how many times it had passed or failed in the past, and whether its status had recently changed. They also fed the system a special score that flagged unusual behavior, helping the model spot anomalies that might indicate a hidden problem. The team trained their new, simpler system on this historical data and then asked it to predict which tests should run first in future scenarios. They compared its performance directly against a more complex, time-aware model that had previously been considered the state-of-the-art solution.

The results were striking. The new, lightweight system performed just as well as the heavy, complex one. In terms of the primary metric used to measure how quickly faults are found, both models achieved identical scores across all three industrial datasets. On one dataset, both models reached a score of 0.959; on another, they both hit 0.893; and on the third, they both scored 0.821. This means that despite being built with a much simpler architecture, the new system was able to rank the tests in the exact same order of effectiveness as the more sophisticated model. The researchers found that the complex model did not offer any advantage in finding faults sooner, nor did it improve the accuracy of the ranking when only a portion of the tests could be run due to time limits.

The true value of this discovery lies in efficiency. The complex model required more than five times as many internal settings, known as parameters, to function compared to the new system. Specifically, the complex model had 54,577 parameters, while the new system needed only 13,057. This massive reduction in size translated directly into speed. The new system trained significantly faster, taking less than half the time per training cycle on some datasets. For example, on one dataset, the new system took 0.719 seconds to process a training step, while the complex model took 1.426 seconds. In an environment where software is updated constantly and models need to be retrained frequently, this speed difference is substantial. It means that companies can update their testing strategies more often without waiting for slow, resource-heavy computers to finish their work.

The study also addressed a specific technical hurdle that often plagues these types of experiments. The researchers discovered that the original complex model, when implemented exactly as described in previous literature, suffered from a flaw where it could only produce a handful of different priority scores, making it less useful. By adjusting the final layer of the model to allow for a continuous range of values, they ensured a fair comparison. Even with this correction, the complex model did not outperform the simple one. The researchers noted that in these specific industrial datasets, the features used to predict failures were so clear that even a simple model could separate the failing tests from the passing ones with perfect accuracy. This suggests that for these types of problems, the extra complexity of remembering long sequences of history was unnecessary.

Ultimately, the work demonstrates that in the specific context of prioritizing tests for continuous integration, a simpler approach can be just as effective as a complex one. The new system, AnoMLP-TCP, proves that you do not need a massive, deep learning architecture to solve this problem. Instead, a streamlined model that focuses on key historical indicators and anomaly detection can deliver the same high-quality results while using a fraction of the computing resources. This finding offers a practical path forward for software teams who need efficient, reliable testing strategies without the burden of maintaining heavy, slow-to-train artificial intelligence systems. The study concludes that while the complex models are not "wrong," they are likely over-engineered for this specific task, and a lighter alternative is available for those who need speed and simplicity.