Technical Summary: LiSA (Lifelong Safety Adaptation via Conservative Policy Induction)

1. Problem Statement

As Large Language Models (LLMs) transition from chat interfaces to autonomous agents capable of accessing private data, invoking privileged tools, and executing multi-step workflows, the cost of safety failures escalates from minor generation errors to concrete harms such as data leaks and unauthorized actions. Existing safety guardrails (e.g., classifiers, rule-based validators, and prompting strategies) rely on static, pre-deployment definitions of harm. However, safety and privacy boundaries are often contextual, shaped by local organizational norms, shifting user expectations, and task-specific risk tolerances that are difficult to enumerate in advance.

This creates a practical gap: guardrails must adapt to their specific operating environments, yet deployment feedback is typically limited to sparse, noisy user-reported failures. Repeated fine-tuning of base models is often impractical in this regime. The paper formulates the problem of deployment-time guardrail adaptation: improving a fixed base guardrail over time using only sparse, potentially noisy user corrections without repeated model retraining.

2. Methodology: LiSA Framework

The authors propose LiSA (Lifelong Safety Adaptation), a conservative policy induction framework organized as an online–offline loop. Instead of fine-tuning the base model, LiSA improves a fixed guardrail through structured policy memory and evidence-aware reuse. The framework consists of three core mechanisms:

A. Structured Policy Memory (Broad Abstraction)

LiSA converts sparse failure reports into reusable policy abstractions rather than storing isolated cases.

• Broad Policy Items: Induced from failure reports, these are natural-language policy statements (e.g., "Sharing general public information is appropriate") accompanied by metadata, including support and contradiction counts.
• Goal: To generalize sparse feedback so that a single correction can guide future decisions in related contexts.

B. Conflict-Aware Local Policies

Broad abstractions can overgeneralize, collapsing nearby contexts with different labels into a single rule.

• Mechanism: LiSA identifies "mixed-label regions" where semantically similar cases have conflicting user reports (e.g., sharing information is appropriate in one context but inappropriate in another).
• Local Rules: In these regions, LiSA induces narrower, label-specific policy items (local rules) that act as refinement cues. Unlike broad policies, these are not subject to confidence gating; they are surfaced immediately to provide fine-grained boundary resolution for the inference model.

C. Confidence-Gated Online Guardrailing

To prevent weakly supported abstractions from influencing inference too early (especially under noisy feedback), LiSA employs a conservative reuse strategy.

• Posterior Lower Bound: For each broad policy item, LiSA models the transfer reliability ($\theta_m$) as a latent accuracy variable with a Beta posterior distribution based on observed support ($s_m$) and contradiction ($c_m$) counts: $\theta_m | \text{data} \sim \text{Beta}(1 + s_m, 1 + c_m)$.
• Gating Rule: A broad item is surfaced only if its confidence, defined as the lower $\delta$-quantile of the posterior, exceeds a label-specific threshold ($\tau_{refuse}$ or $\tau_{allow}$).
• Rationale: This ensures that broad policies must accumulate sufficient evidence before influencing decisions, preventing brittle rules derived from single reports from causing over-generalization.

D. Online-Offline Loop

• Online Phase: The system retrieves relevant broad and local policies, filters broad items via confidence gating, and prompts the base guardrail with the surviving context. If no policies apply, it falls back to the base guardrail.
• Offline Phase: Periodically, accumulated user reports are used to rebuild the memory. This involves inducing new broad policies, merging semantically overlapping items, regenerating local rules for mixed-label regions, and updating confidence statistics.

3. Key Contributions

1. Problem Formulation: The paper defines the setting of lifelong guardrail adaptation under sparse, noisy supervision, distinguishing it from standard supervised updating or dense fine-tuning.
2. LiSA Framework: A structured policy memory system featuring:
   • Broad policy abstraction for sparse reuse.
   • Conflict-aware local refinement to handle mixed-label regions.
   • Conservative confidence-gated reuse based on a Beta-posterior lower bound to control the surfacing of broad policies.
3. Empirical Validation: Demonstrations across three benchmarks (PrivacyLens+, ConFaide+, and AgentHarm) showing that LiSA:
   • Outperforms strong memory-based baselines (e.g., ReasoningBank, Synapse, AGrail) under sparse feedback.
   • Remains robust to noisy user reports (up to 20% label-flip rates), with confidence gating identified as the key stabilizing factor.
   • Improves boundary-sensitive decisions through local refinement.
   • Pushes the latency–performance frontier beyond what is achievable by simply scaling the base model size.

4. Experimental Results

• Performance: Across PrivacyLens+, ConFaide+, and AgentHarm, LiSA consistently achieved higher macro-F1 scores than fixed base guardrails and baselines using either case retrieval (Synapse) or broad policy abstraction alone (ReasoningBank).
• Ablation Studies:
  • Removing local rules caused the largest performance drop, confirming their necessity for resolving boundary ambiguities.
  • Removing confidence gating had a smaller effect on mean performance but significantly increased variance and reduced robustness under noise.
• Noise Robustness: Under 20% label noise, LiSA maintained high performance (0.917 F1), whereas baselines like ReasoningBank dropped significantly (0.857 F1). The Beta-posterior gating effectively prevented weakly supported policies from dominating inference.
• Latency vs. Performance: LiSA attached to a lightweight model (Gemini-3.1-flash-lite) outperformed larger, un-adapted backbones (e.g., Flash, Pro) in terms of the F1-latency trade-off. The offline cost of memory refresh is amortized over many decisions, making structured memory more efficient than model scaling for sparse-feedback adaptation.

5. Significance and Claims

The paper claims that effective adaptation for AI agents does not require fine-tuning or larger models but rather calibrated reuse of deployment experiences. LiSA offers a practical path to secure AI agents against the unpredictable "long tail" of real-world edge risks by:

• Allowing guardrails to learn from their operating environments without repeated retraining.
• Maintaining stability through conservative, evidence-grounded caution.
• Providing a more efficient route to improved safety than simply scaling backbone model capacity.

The authors position LiSA as a "practical middle ground" that avoids the brittleness of static guardrails and the risks of unconstrained adaptation, ensuring that guardrails evolve in response to local norms while preserving safety through evidence-calibrated mechanisms.