💻 computer science

Read, Extract, Classify: A Tool for Smarter Requirements Engineering

This paper introduces ReXCL, an automated tool that enhances requirements engineering by extracting data from semi-structured documents and classifying requirements using predictive modeling and adaptive fine-tuning, thereby significantly improving efficiency and accuracy in the software development lifecycle.

Original authors: Paheli Bhattacharya, Manojit Chakraborty, Santhosh Kumar Arumugam, Rishabh Gupta

Published 2026-05-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Paheli Bhattacharya, Manojit Chakraborty, Santhosh Kumar Arumugam, Rishabh Gupta

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a librarian trying to organize a massive, chaotic pile of customer letters. Some are written on fancy letterheads, some are scribbled on napkins, and some are just long, messy paragraphs. Your job is to read every single one, figure out which part is the "subject line," which part is the "body," and then sort them into four specific bins: Info, Header, Functional Request (what the product must do), and Non-Functional Request (how the product should perform).

Doing this by hand is slow, boring, and prone to mistakes. That's where the ReXCL tool comes in. Think of ReXCL as a super-smart, automated robot librarian designed specifically for the automotive industry.

Here is how it works, broken down into two main steps:

Step 1: The "Extraction" Robot (Cleaning and Organizing)

First, the robot takes a messy document (like a PDF or Word file) and tries to make sense of the layout.

The Noise Filter: Imagine the document has page numbers, dates, and "Page 1 of 5" stamped all over the top and bottom. These are distractions. The robot uses a simple, fast brain (called a Random Forest classifier) to spot these repetitive patterns and throw them away, just like a sieve separates sand from gold.
The Structure Builder: Once the noise is gone, the robot looks for the "skeleton" of the document. It finds the section numbers (like "1.1") and the titles (like "Main Task"). It then grabs the text that belongs under those titles.
The Result: It turns the messy document into a neat, structured table. It's like taking a jumbled box of LEGO bricks and snapping them together into a clear instruction manual where every piece has its own labeled slot.

Step 2: The "Classification" Robot (Sorting and Labeling)

Now that the text is clean and organized, the second robot takes over. This one is much smarter and has been specially trained.

The Specialist Training: Imagine a general language expert (like a standard BERT model) who knows English well but doesn't know much about cars. This robot takes that expert and gives them a crash course in "Automotive Speak" by reading thousands of unlabeled car requirement documents. This is called Adaptive Fine-Tuning. It's like teaching a general chef to specialize in Italian cuisine by having them taste and study thousands of Italian recipes before they start cooking.
The Sorting: Now, the robot reads each piece of text and asks, "Is this a fact? Is this a header? Is this a rule about what the car must do (Functional), or a rule about how fast it should go (Non-Functional)?"
The Result: It slaps a digital label on every sentence. The paper claims this robot is incredibly accurate, especially for tricky categories that other tools miss.

The Proof: Did it work?

The researchers tested their robot against human experts.

For Extraction: Three experts checked the robot's work on 445 sentences. The robot got a score of 4.45 out of 5. It was also 96% accurate at spotting and removing those annoying page headers and footers.
For Classification: When the robot tried to sort the text, it was a massive improvement over a standard, untrained model.
- A standard model struggled, getting scores as low as 0.22 (out of 1.0) for difficult categories.
- The ReXCL robot, after its special training, scored between 0.93 and 0.99. It was almost perfect at telling the difference between a "Functional Requirement" and a "Non-Functional Requirement."

The Big Picture

The paper concludes that ReXCL is a tool that automates the boring, error-prone work of turning messy customer documents into clean, structured data.

What it does: It takes raw files, cleans them, structures them, and labels them.
What it doesn't do (yet): It currently handles text documents. The authors mention that in the future, they want to teach it to understand images, tables, and spreadsheets, but right now, it sticks to text.
The Output: Once the robot is done, you can download the results as a neat file (like Excel or CSV) and feed it directly into professional engineering tools like IBM DOORS.

In short, ReXCL is a digital assistant that turns a chaotic pile of requirement documents into a perfectly organized, labeled filing cabinet, saving engineers from hours of manual sorting.

Technical Summary of ReXCL: A Tool for Smarter Requirements Engineering

Problem Statement
In software development, effective Requirements Engineering (RE) is critical for aligning system outputs with stakeholder expectations. However, the current workflow for handling customer requirement specifications—typically provided in semi-structured formats like PDF or DOC files—is largely manual, error-prone, and labor-intensive. Stakeholders must manually interpret diverse document layouts, extract section structures, map content into predefined schemas, and categorize requirements (e.g., functional, non-functional, informational). These manual steps introduce delays, reduce traceability, and compromise the overall quality of the RE process, particularly when integrating data into downstream tools like IBM DOORS.

Methodology
To address these challenges, the authors present ReXCL, a tool designed to automate both the extraction and classification of requirement texts. The system operates through two primary modules:

Extraction Module:
- Input Processing: The module transforms raw input documents (PDF, DOC, TXT) into an intermediate markdown-based text representation using packages like PyMuPDF4LLM. This approach preserves document structure, including section titles (marked with special tokens) and table formatting, unlike standard text converters that flatten structure.
- Noise Removal: To eliminate headers and footers (e.g., page numbers, dates), a lightweight Random Forest classifier is employed. This binary classifier uses two features—frequency of occurrence and positional information—to identify and remove redundant sentences located at the top or bottom of pages.
- Structural Parsing: The cleaned text is parsed to identify section numbers, headings, and associated text blocks. These are grouped into structured tuples of the form $\langle \text{number}_i, \text{heading}_i, [\text{text}_{i1}, \dots, \text{text}_{im}] \rangle$ .
- Output: The tuples are formatted into a tabular schema containing Object Identifiers, Numbers, Headings, Text, and Levels, ready for downstream processing.
Classification Module:
- Domain Adaptation: The system utilizes a BERT-based model customized for the automotive domain. The process begins with Iterative Masked Language Modeling on large-scale, unlabeled automotive requirement texts. This step refines the pre-trained general-purpose model to capture domain-specific terminology and syntactic structures.
- Supervised Fine-Tuning: Following adaptation, the model is fine-tuned in a supervised setting using labeled requirement texts. A multi-layer perceptron (MLP) is applied to the [CLS] token of the transformer's output to perform document-level semantic classification.
- Categorization: The model assigns each requirement text to one of four categories: Info, Header, Functional Requirement, or Non-Functional Requirement.

Key Contributions

Automated Schematization: ReXCL offers a novel approach to automating the schematization of semi-structured requirement documents, a task previously requiring significant human effort.
Integrated Pipeline: Unlike prior work that addressed header detection or section classification separately, ReXCL provides a comprehensive pipeline that handles both structural extraction and semantic classification.
Adaptive Fine-Tuning: The tool demonstrates the efficacy of a two-phase training process (domain adaptation followed by supervised fine-tuning) for handling low-resource classification tasks in the automotive domain.
Interoperability: The final output is exportable in standard formats (CSV, Excel, YAML, JSON) and is compatible with industry-standard RE tools such as IBM DOORS and Jira.

Evaluation Results

Extraction Performance: In a manual evaluation involving three domain experts reviewing 445 sentences, the extraction module achieved an average accuracy score of 4.45/5. The header/footer removal component achieved an average accuracy of 96.05%.
Classification Performance: Empirical results show substantial gains from adaptive fine-tuning compared to a baseline "vanilla" BERT model.
- Vanilla BERT F1-scores: Ranged between 0.22 and 0.41 across categories.
- Adaptive Finetuned BERT F1-scores: Achieved 0.99 for Header, 0.98 for Info, 0.98 for Functional Requirements, and 0.93 for Non-Functional Requirements.
- The improvements were particularly significant for under-represented categories like Non-Functional Requirements.

Significance and Claims
The paper claims that ReXCL significantly improves the efficiency and accuracy of managing requirements, marking a shift from manual, error-prone processes to automated, data-driven workflows. By reducing human effort and increasing consistency, the tool streamlines the transformation of raw documents into structured, actionable data. The authors position ReXCL as a practical solution for real-world settings, validated by expert review and empirical metrics.

Future Work
The authors note that future iterations of the tool aim to extend support to images, tables, and multi-modal documents, including presentations and spreadsheets.

Step 1: The "Extraction" Robot (Cleaning and Organizing)

Step 2: The "Classification" Robot (Sorting and Labeling)

The Proof: Did it work?

The Big Picture

Technical Summary of ReXCL: A Tool for Smarter Requirements Engineering

More like this