TriagerX: Dual Transformers for Bug Triaging Tasks with Content and Interaction Based Rankings

1. Problem Statement

Bug triaging involves assigning reported software issues to the most suitable developer or team. While traditional Machine Learning (ML) and Information Retrieval (IR) methods (e.g., TF-IDF, Bag of Words) have been used, they often fail to capture the nuanced semantic context of bug reports. Recent Pre-trained Language Models (PLMs) like BERT and RoBERTa offer better semantic understanding but suffer from specific limitations in industrial settings:

• Token Attention Issues: PLMs may attend to less relevant tokens within a bug report, reducing effectiveness.
• Lack of Interaction Context: Standard PLM approaches often ignore the historical interaction history of developers (e.g., who recently fixed similar bugs), which is a strong signal in real-world workflows.
• Model Constraints: Single large PLMs can be computationally expensive, prone to catastrophic forgetting during fine-tuning, or constrained by the generalization limits of knowledge distillation (e.g., in models like LBT-P).
• Cold Start & Turnover: Existing methods struggle when developer roles change or when new developers join without sufficient history.

2. Methodology: The TriagerX Framework

TriagerX is a hybrid framework that combines Content-Based Ranking (CBR) and Interaction-Based Ranking (IBR) to address the limitations of existing state-of-the-art (SOTA) models.

A. Content-Based Ranker (CBR)

The CBR component focuses on the textual semantics of the bug report.

• Dual-Transformer Architecture: Instead of a single PLM, TriagerX ensembles two distinct PLMs (specifically DeBERTa-base and RoBERTa-base). This leverages the orthogonality and complementary strengths of different pre-trained models.
• Multi-Layer Feature Extraction: Unlike standard approaches that use only the final [CLS] token, CBR extracts hidden states from the last three layers of both transformers. This captures a broader range of syntactic and semantic features.
• Learnable Weighting: The model uses learnable weights to adaptively combine features from different layers and different PLMs.
• CNN-Based Classification: Instead of simple Fully Connected Networks (FCNs), CBR employs Convolutional Neural Networks (CNNs) with variable filter sizes ($3, 4, 5, 6$) to capture local patterns and n-gram features from the concatenated embeddings.
• Knowledge Preservation Fine-Tuning (KPFT): To prevent catastrophic forgetting, the initial layers of the PLMs are frozen, while the final layers are fine-tuned. This balances retaining general language knowledge with adapting to specific bug triaging tasks.

B. Interaction-Based Ranker (IBR)

The IBR component refines recommendations based on developer history.

• Similarity Retrieval: It uses Siamese BERT (SBERT) to retrieve semantically similar past issues from the repository.
• Time-Decayed Scoring: It calculates an interaction score for developers based on their involvement (commits, PRs, discussions) with similar issues. Crucially, it applies an exponential decay function ($e^{-\lambda t}$) to weight recent contributions higher than older ones.
• Interaction Points: Different interaction types are assigned different weights (e.g., Commits/PRs > Discussions > Assignments).
• Normalization: Scores are normalized to ensure comparability across developers.

C. Rank Aggregator (RAgg)

The final recommendation is generated by fusing the scores from CBR and IBR using a Weighted Ranking Aggregation (WRA) method:
$$FS(d_i) = NPS(d_i) + W_f \cdot NIS(d_i)$$
Where $NPS$ is the normalized prediction score from CBR, $NIS$ is the normalized interaction score from IBR, and $W_f$ is a tunable weight factor optimized via grid search for each dataset.

3. Key Contributions

1. Dual-Transformer Ensemble: Proposes a novel CBR that ensembles two PLMs and extracts multi-layer representations, outperforming single large PLMs while using fewer total parameters.
2. Interaction-Aware Hybrid Framework: Introduces the first bug triaging method that integrates a time-sensitive, context-aware interaction ranking (IBR) with a PLM-driven content ranking.
3. Industrial Deployment: Successfully deployed TriagerX within IBM's OpenJ9 development environment, demonstrating low latency and high acceptance in a real-world setting.
4. Component Recommendation: Extended the framework to recommend software components (acting as team proxies), addressing scenarios where direct developer assignment is difficult due to turnover or unavailability.

4. Experimental Results

The authors evaluated TriagerX on five datasets: three literature benchmarks (Google Chromium, Mozilla Core, Mozilla Firefox) and two new industrial benchmarks (OpenJ9, TypeScript).

• Performance vs. Baselines: TriagerX consistently outperformed nine SOTA baselines, including LBT-P, DBRNN-A, MDN, and various large PLMs (BERT-large, RoBERTa-large, CodeBERT).
  • OpenJ9: Achieved a 54% improvement in Top-1 accuracy over LBT-P and a 58% improvement over the nearest large PLM baseline.
  • TypeScript: Achieved a 26% improvement over LBT-P and a 10% improvement over the nearest large PLM.
  • Literature Datasets: Improved Top-1 accuracy by up to 22% on literature datasets and 29% on partner datasets compared to baselines.
• Ablation Studies:
  • Ensemble Effect: Combining DeBERTa and RoBERTa significantly improved accuracy over using them individually, confirming the value of model orthogonality.
  • Classifier Choice: The CNN-based classifier significantly outperformed FCNs (e.g., >80% improvement in Top-1 on OpenJ9).
  • Hyperparameters: Optimizing the time-decay factor ($\lambda$) and similarity threshold ($\tau$) was critical; positive $\lambda$ values (prioritizing recent activity) yielded the best results.
• Efficiency: Despite high accuracy, TriagerX CBR uses fewer parameters (~271M) than large PLMs (e.g., DeBERTa-Large at 405M) and achieves inference times of ~26ms on GPUs.

5. Significance and Impact

• Practical Viability: The deployment at IBM demonstrates that advanced deep learning models can be integrated into existing CI/CD workflows (via GitHub Actions and Slack) with minimal latency (3-4 seconds on CPU).
• Addressing Real-World Constraints: By incorporating interaction history and component-level recommendations, TriagerX addresses practical issues like developer turnover and team reassignments, which pure content-based models miss.
• Cold Start Mitigation: While not fully solving the cold-start problem, the component recommendation feature allows issues to be routed to the correct team, enabling team leads to assign new developers who can then build a history for future automated recommendations.
• Trust and Explainability: The framework provides a starting point for human triagers, who can weigh the AI's suggestion against contextual factors (e.g., workload), fostering a "human-in-the-loop" approach rather than full automation.

In conclusion, TriagerX represents a significant advancement in automated bug triaging by effectively merging deep semantic understanding with dynamic developer activity signals, achieving superior accuracy and industrial applicability.