IssueGuard: Real-Time Secret Leak Prevention Tool for GitHub Issue Reports

1. Problem Statement

Developers frequently handle sensitive credentials (API keys, OAuth tokens, private keys) within collaborative platforms like GitHub and GitLab. While secret detection is common in source code, a significant risk exists in unstructured text found in issue reports, logs, and documentation.

• The Gap: Existing tools (e.g., TruffleHog, Gitleaks) typically operate post-submission (scanning repositories after commits) or within CI/CD pipelines. This reactive approach fails to prevent the initial exposure of secrets in public issue trackers.
• The Challenge: Current real-time solutions lack context. Traditional regex-based scanners suffer from high false positive rates (flagging placeholders or dummy keys as secrets), leading to "alert fatigue" and poor developer adoption. There is no existing tool that provides real-time, pre-submission feedback specifically within the issue-writing interface with high precision.

2. Methodology

IssueGuard is a browser-based solution designed to integrate seamlessly into the developer workflow. It employs a client-server architecture and a two-stage detection pipeline.

System Architecture

• Client: A Google Chrome extension that monitors the issue description editor on GitHub and GitLab.
• Server: A FastAPI-based backend service that handles the heavy computational load of model inference.
• Workflow Optimization:
  • Debouncing: The client waits for a pause in user typing before sending text to the server to avoid excessive network requests.
  • Asynchronous Processing: The server uses a thread pool and mixed-precision (FP16) inference on GPU to ensure low latency.
  • Caching: A Least Recently Used (LRU) cache stores results for repeated text patterns to improve responsiveness.

Detection Pipeline

1. Candidate Extraction: Upon receiving text, the backend applies 761 regular expressions to identify potential secret strings (e.g., patterns resembling API keys).
2. Contextual Classification:
   • Unlike simple regex scanners, IssueGuard extracts a 200-character context window surrounding each candidate string.
   • This context is concatenated with the candidate and fed into a fine-tuned CodeBERT model (a transformer-based model pre-trained on code and natural language).
   • The model classifies the candidate as either "Secret" or "Non-sensitive" (e.g., distinguishing a real AWS key from a placeholder like YOUR_API_KEY).
3. User Feedback: Confirmed secrets are highlighted in red within the editor with a tooltip warning, allowing the developer to correct the issue before submission.

3. Key Contributions

1. IssueGuard Tool: The first browser extension to integrate a transformer-based secret detection model directly into GitHub/GitLab issue editors, enabling pre-submission leak prevention.
2. Hybrid Detection Approach: A novel pipeline combining regex extraction with a fine-tuned CodeBERT classifier to significantly reduce false positives while maintaining high recall.
3. Comprehensive Evaluation:
   • Benchmarking: Comparison against state-of-the-art Large Language Models (LLMs) and traditional scanners.
   • Real-World Validation: Testing on 178 real-world GitHub repositories.
   • Usability Study: A study involving 50 participants (developers, testers, managers) to assess workflow integration and user confidence.

4. Results and Performance

Model Effectiveness

• Accuracy: IssueGuard achieved an F1-score of 92.70% on the benchmark dataset.
• Comparison: It outperformed larger models like StarCoder (15B) (F1: 89.01%) and specialized PII detectors like StarPII (F1: 90.24%).
• Precision vs. Recall: IssueGuard achieved 92.49% Precision and 92.91% Recall. In contrast, traditional tools like TruffleHog had high recall (94%) but low precision (55%), which would cause excessive false alarms in a real-time setting.
• Generalization: On 178 real-world repositories, the model maintained a macro-average F1-score of 81.60%.

Latency and Performance

• End-to-End Latency: The average time from typing completion to visual highlighting is 198 milliseconds (0.198s).
• Inference Speed: The core model inference step takes only 10.1 milliseconds, proving the architecture is suitable for uninterrupted real-time use.

User Study

• Participants: 50 users with varying experience levels (0–10+ years).
• Confidence: 90% of participants rated their confidence in the tool as 4 or 5 (out of 5).
• Satisfaction: 80% reported being "Very Satisfied" with ease of use, and 75% with the tool setup.
• Qualitative Feedback: Users noted that the "native" feel of the tool reduced cognitive load and significantly increased confidence in submitting public issues without accidental leaks.

5. Significance

• Shift from Reactive to Proactive: IssueGuard moves secret detection from a post-commit audit phase to the moment of creation, preventing secrets from ever entering the public record.
• Contextual Intelligence: By leveraging CodeBERT, the tool solves the "false positive" problem that plagues regex-based scanners, making it practical for daily developer use without causing alert fatigue.
• Developer-Centric Security: The tool integrates directly into existing workflows (GitHub/GitLab) without requiring developers to change their habits or run external scans, lowering the barrier to secure coding practices.
• Privacy Preservation: The architecture allows for local execution or controlled backend deployment, addressing privacy concerns associated with sending code snippets to cloud-based security services.

In conclusion, IssueGuard demonstrates that high-accuracy, real-time secret detection is feasible and essential for protecting unstructured text in collaborative development environments.