Technical Summary: SingGuard-NSFA

Problem Statement

The rapid evolution of Large Language Models (LLMs) into autonomous agents has fundamentally shifted the security threat surface. Unlike conversational AI, where risks are primarily confined to content safety (e.g., toxic output or misinformation), agentic AI systems execute real-world actions such as browsing the web, invoking APIs, and running code. This transition introduces operational threats where a security breach can lead to sensitive data exfiltration, unauthorized tool manipulation, and remote code execution. Existing guardrails face three critical limitations:

1. Lack of Systematic Taxonomy: Current risk definitions are often fixed, scenario-specific, or entirely user-defined, lacking a principled framework to assess coverage.
2. Extensibility Bottlenecks: Adapting detection to new risks typically requires manual reconfiguration or full model retraining, which is inefficient and unreliable.
3. Language Bias: Existing evaluations are predominantly English-centric, leaving exploitable blind spots for non-English evasion.

Furthermore, current safety resources focus on "what a model says" rather than "what an agent does," failing to address the CIA (Confidentiality, Integrity, Availability) triad in the context of agent operations.

Methodology

SingGuard-NSFA is a guardrail framework designed to secure agentic AI systems through three tightly integrated components: a risk taxonomy, a multilingual benchmark suite, and a dual-mode detection architecture.

1. NSFA Risk Taxonomy

The authors introduce the NSFA (Not-Secure-For-Agents) taxonomy, a hierarchical classification framework grounded in the CIA triad and cross-validated against three OWASP guidelines (OWASP Top 10 for LLM Applications, OWASP Agentic AI Threats, and OWASP Top 10 for Agentic Applications).

• Structure: It organizes 185 risk variants into 7 Level-1 domains, 28 Level-2 risks, and 185 Level-3 variants.
• Scope: The taxonomy distinguishes between Query-side threats (5 domains, 160 variants) and Response-side threats (2 domains, 25 variants).
• Design Principles:
  • Query-First, Response-as-Backstop: Intercepts threats at the input stage to prevent irreversible actions, with response detection acting as a secondary layer.
  • Single-Turn Detectability: Focuses on risks identifiable from a single turn of text (query or response), excluding multi-turn trajectory analysis to ensure stateless, low-latency deployment.
  • Multilingual Coverage: Designed to prevent language-based evasion across 133 languages.

2. Benchmark Suite Construction

To evaluate guardrails against the NSFA taxonomy, the authors constructed a comprehensive benchmark suite:

• Purpose-Built Benchmarks: Generated via a four-stage pipeline (seed-free generation, seed-based augmentation, multilingual expansion, and verification) using 74 open-source LLMs. This includes 63,431 query samples and 29,972 response samples, covering all 185 risk variants with balanced positive/negative distributions.
• CrossSource Benchmark: Adapted from five public agent-security datasets (AgentDojo, InjecAgent, AgentHarm, AgentDyn, ATBench), normalized into single-query formats, and expanded to 132 languages, resulting in 3,435 samples.
• Total Coverage: The suite spans 133 languages and includes over 93,000 purpose-built samples.

3. SingGuard-NSFA Framework

The framework employs a dual-mode approach combining generative reasoning and discriminative classification:

• Generative Reasoning (Offline Auditing): A Supervised Fine-Tuned (SFT) backbone (based on Qwen3.5 models) generates interpretable chain-of-thought (CoT) analyses followed by risk judgments. This mode supports human review and incident investigation.
• Real-Time Classification (Online Interception): Lightweight discriminative Multi-Layer Perceptron (MLP) classification heads are trained on the frozen SFT backbone's last-token embeddings. These heads output risk probabilities for each Level-1 domain in a single forward pass, enabling detection latency of approximately 45–57 ms.
• Extensibility: The classification-head architecture is natively extensible. Adding detection for new risks requires training only a new lightweight head on the frozen embeddings, without retraining the backbone or disrupting existing capabilities.

Key Contributions

1. NSFA Taxonomy: A systematic, CIA-triad-grounded classification of 185 agent-specific risk variants, validated against three major OWASP frameworks, explicitly separating attack techniques from objectives.
2. Multilingual Benchmark Suite: The largest and most linguistically diverse evaluation suite for agent guardrails to date, covering 133 languages with over 93K purpose-built samples and a cross-source benchmark.
3. Dual-Mode Guardrail Architecture: A framework that unifies interpretable generative reasoning for auditing with low-latency discriminative classification for real-time interception. The classification-head design allows for plug-in extensibility to new risks and other guardrail models.
4. Synthetic Data Pipeline: A robust four-stage pipeline utilizing 74 open-source LLMs to generate diverse, multilingual training data with hard-negative contrasts and rigorous verification.

Experimental Results

The authors evaluated four SingGuard-NSFA models (0.8B, 2B, 4B, and 9B parameters) against 10 competing guardrails (including Llama Guard 3, Granite Guardian, and AgentDoG) across three benchmarks.

• Binary Detection Performance:
  • On purpose-built benchmarks, all SingGuard-NSFA models achieved ≥94% F1, surpassing the strongest competing guardrail (AgentDoG1.5-FG) by 6 to 12 absolute F1 points.
  • The 9B model achieved 96.95% F1 on Query and 98.31% F1 on Response benchmarks.
  • On the CrossSource-Query benchmark, the 9B model attained 91.29% F1, outperforming the best competitor (84.67%) by 6.6 points, with a more balanced precision-recall trade-off.
• Inference Latency:
  • The real-time classification mode operates at 45–57 ms per sample, comparable to the fastest competing guardrails (14–52 ms) despite using significantly larger models (up to 9B parameters).
  • The generative reasoning mode takes 3.6–7.2 seconds, suitable for offline auditing.
• Extensibility:
  • Ablation studies showed that scaling the number of classification heads from 5 to 50,000 increased end-to-end latency by only 9 ms.
  • Training a content safety head on the frozen SingGuard-NSFA backbone allowed it to outperform dedicated content safety guardrails on standard moderation benchmarks.
  • Applying the classification-head approach to a frozen Llama Guard 3 backbone improved its F1 score by 17.6 points on the Query benchmark, demonstrating the architecture's generalizability as a plug-in enhancement.

Significance

The paper posits that SingGuard-NSFA addresses the critical gap between content safety and operational agent security. By formalizing a risk taxonomy grounded in the CIA triad and providing a scalable, multilingual detection framework, it offers a principled approach to securing agentic AI. The dual-mode architecture allows for both high-throughput real-time interception and deep, interpretable auditing. The demonstrated extensibility of the classification-head architecture suggests a pathway for guardrails to evolve alongside emerging threats without the prohibitive cost of full model retraining. The authors conclude that these results demonstrate the feasibility of addressing content safety and operational agent security within a unified, extensible framework.