ARQM: A Tool for Requirements Quality Analysis

Imagine you are the editor of a massive, chaotic library where every book is a blueprint for building something new—a bridge, a video game, or a spaceship. Before a single brick is laid or a line of code is written, the architects must write down exactly what they need. These notes are called "requirements." But here's the catch: human beings are terrible at writing these notes perfectly. We get tired, we use vague words like "maybe" or "fast," and sometimes we accidentally write things that contradict each other. If the blueprint is messy, the building collapses, or the game crashes. This is the world of Requirement Engineering: the messy, high-stakes process of turning human ideas into clear, strict instructions for machines.

To fix this, scientists have been building digital "proofreaders" for decades. Traditionally, these proofreaders were like strict librarians with a red pen and a list of simple rules: "If you see the word 'must,' it's a requirement. If you see the word 'process,' it's not." But these old rules often missed the nuance of human language. Enter Artificial Intelligence (AI) and Machine Learning (ML). Think of these not as rule-followers, but as super-observant apprentices who have read millions of documents. Instead of just looking for specific words, they learn the vibe of a sentence, understanding context and meaning in a way that simple lists never could. The big question researchers are asking is: Can these smart apprentices do a better job than the old rule-based librarians at spotting bad blueprints before construction even begins?

This paper introduces a new digital apprentice named ARQM (Automated Requirements Quality Management). The researchers from Pennsylvania State University built this tool to act as a tireless, hyper-attentive editor for requirement documents. Unlike its predecessors, which relied on outdated, rigid rules, ARQM uses modern AI to "read" documents and perform two main tasks. First, it acts as a screening filter, sorting through a jumbled pile of text to decide, "Is this sentence actually a requirement, or is it just background noise?" Second, it acts as a quality inspector, checking if the requirements that pass the filter are actually good. It looks for specific "smells" or violations based on the IEEE 29148 standard—a global rulebook for what makes a requirement clear, testable, and unambiguous.

The team didn't just build the tool; they put it through a rigorous training camp. They fed it thousands of sentences from a public dataset called PURE (PUblic REquirements) to teach it how to spot requirements. Then, they tested it on a special set of documents that had been carefully labeled by human experts to check for four specific types of quality issues: Ambiguity (is it confusing?), Feasibility (can it actually be done?), Singularity (does it try to do too many things at once?), and Verifiability (can we prove it works?).

The results suggest that the AI apprentice is indeed a powerful new hire. When it came to simply identifying what is and isn't a requirement, the tool's most advanced model, DistilBERT, performed better than almost every other method tested, achieving a high level of accuracy. It learned that requirements often sound formal and rigid (using words like "shall" or "must"), while non-requirements often sound like administrative chatter. However, the paper notes that the AI isn't magic; it still struggles a bit with the most subtle, human-like nuances. For instance, while the AI got very good at spotting requirements that were "singular" (focused on one thing), it found it harder to agree with humans on what counted as "ambiguous" or "verifiable." This suggests that while AI can catch the obvious mistakes, the trickiest, most subjective errors still require a human touch.

One of the coolest features of ARQM is its use of Explainable AI (XAI). Instead of just saying, "This sentence is bad," the tool highlights the specific words that made it say so. It's like a teacher circling the exact phrase in red and writing, "This word is too vague." This helps the human writer understand why their requirement needs fixing, rather than just getting a cryptic error message.

The researchers are careful to point out that their tool isn't a perfect, solved problem yet. They found that their training data had some gaps; for example, there were very few examples of "unfeasible" requirements, which made it hard for the AI to learn that specific lesson. They also noted that the human experts who labeled the data didn't always agree with each other, which means the AI sometimes had to learn from conflicting instructions. Despite these limitations, the study suggests that modern AI methods are a significant step forward. They can learn general patterns that old rule-based tools missed, offering a faster, more holistic way to clean up the blueprints of our digital world. The paper concludes that while ARQM is a strong start, the future of this technology lies in training it on even more diverse data and helping it understand the deeper context of the projects it's helping to build.