Technical Summary: Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification

1. Problem Statement

Large Language Model (LLM)-based Multi-Agent Systems (MAS) have emerged as powerful tools for collaborative decision-making and complex problem-solving. However, their reliance on frequent inter-agent communication introduces significant security vulnerabilities. While prior research has largely focused on independent adversarial behaviors (where malicious agents act in isolation) or predefined coordination patterns, there is a critical gap in understanding and defending against adaptive cooperative attacks.

In such scenarios, malicious agents autonomously coordinate through multi-round interactions, exchanging internal information to dynamically adjust their strategies. This collaboration allows them to fabricate semantically coherent but factually incorrect misinformation that is repeatedly reinforced, leading to a more severe degradation of system performance than independent attacks. Existing defense mechanisms often operate at the agent level, making it difficult to identify localized misinformation or explicitly correct erroneous content, resulting in limited interpretability.

2. Methodology

The paper proposes a dual-pronged approach: a novel Cooperative Attack Method to simulate potent threats and the STAR (Sentence-Level Trustworthiness Analysis and Rectification) framework to defend against them.

2.1 Adaptive Cooperative Attack Method

To simulate a more potent threat, the authors introduce an attack framework where multiple malicious agents ($C \subset A$) controlled by an adversary dynamically coordinate. This method relies on two mechanisms:

• Ally-Context Awareness: Malicious agents maintain a context ($H_C$) containing the historical responses of their malicious allies. This allows them to assess the collective attack posture.
• Cooperative Response Formulation: An LLM-based Judge ($J$) evaluates the persuasive strength of the ally context. Based on this judgment, the agent dynamically selects one of two strategies:
  • Reinforcement: If the ally context is strong, the agent explicitly endorses allied arguments to fabricate a false consensus.
  • Innovation: If the ally context is weak, the agent introduces new, robust perspectives from a novel angle to persuade benign agents.

2.2 STAR Defense Framework

STAR is a training-free, plug-and-play defense framework designed to scrutinize agent communications sentence-by-sentence. It operates through four stages:

1. Sentence-Level Decomposition and Verification:

   • Agent responses are decomposed into a sequence of sentences.
   • An LLM-based Verifier ($V$) processes all sentences simultaneously to perform Factual Assessment (determining accuracy and confidence score $\rho$) and Error Correction (generating specific corrective content $e$ for errors).

2. Suspicion Modeling via Cumulative Confidence:

   • A Suspicion Score ($\Omega$) is calculated for each agent by accumulating the confidence scores of sentences identified as factual errors.
   • This allows for the identification of agents who may appear mostly correct but contain subtle, critical fabrications.

3. Targeted Rectification:

   • Agents exceeding a detection threshold ($\tau$) are flagged as suspicious.
   • A Rectification module ($R$) integrates the specific error corrections generated in step 1 into the original text. This not only neutralizes the threat but also alerts the system to specific misinformation patterns, preserving factually correct information within the same response.

4. Robust Decision Aggregation:

   • The final answer is determined via majority voting, but the identified malicious agents are explicitly excluded from the voting pool to prevent skewing the outcome.

3. Key Contributions

• Cooperative Attack Method: The introduction of an adaptive attack framework where malicious agents dynamically coordinate strategies through ally-aware interactions, demonstrating significantly higher disruption capabilities than independent or static collusive attacks.
• Sentence-Level Defense Framework (STAR): A novel, training-free defense mechanism that performs granular, sentence-level analysis to explicitly localize, correct, and mitigate misinformation, offering high interpretability compared to agent-level filtering.
• Extensive Empirical Evaluation: Comprehensive experiments across diverse datasets (MMLU, CSQA, LogiQA) and LLM backbones (Qwen-Plus, GPT-3.5-Turbo, GPT-5-Nano) validating the severity of cooperative attacks and the efficacy of STAR.

4. Experimental Results

The authors conducted experiments on 400 instances per dataset with a 5-agent MAS (3 benign, 2 malicious).

• Impact of Cooperative Attacks: Cooperative attacks caused a significantly larger degradation in Task Success Rate (TSR) compared to independent attacks. Across all settings, cooperative attacks resulted in a relative drop of 5.34% in TSR and a relative increase in Attack Success Rate (ASR). On the MMLU dataset, cooperative attacks reduced the average TSR by 6.17% and increased ASR by 7.00% compared to the independent baseline.
• Effectiveness of STAR:
  • STAR effectively mitigated both cooperative and independent threats, improving the average TSR by 36.76% compared to the no-defense baseline.
  • It reduced the average ASR by 45.17%.
  • STAR outperformed baseline defenses (G-Safeguard, BlindGuard, ARGUS, Challenger&Inspector) across most configurations.
  • The detection accuracy for malicious agents consistently exceeded 70% across all datasets and models.
• Ablation Study: Removing the Targeted Rectification module caused the most significant performance degradation, highlighting the importance of sentence-level correction. Removing Robust Decision Aggregation also led to noticeable drops, confirming the value of excluding malicious agents from final voting.
• Cost Analysis: While STAR incurs a higher token cost (2.03× relative to no defense) compared to lighter methods like C&I (1.40×), it remains significantly more efficient than complex reasoning-based defenses like ARGUS (7.66×).

5. Significance and Claims

The paper claims that the proposed Cooperative Attack Method reveals a critical, previously under-explored vulnerability in MAS: the ability of malicious agents to adaptively coordinate and reinforce misinformation, posing a significantly amplified threat compared to isolated attacks.

Consequently, the STAR framework is presented as a significant advancement in MAS security. By shifting the defense paradigm from coarse-grained agent-level filtering to fine-grained, sentence-level analysis and rectification, STAR provides:

1. Enhanced Robustness: Effective mitigation against both independent and coordinated adaptive attacks.
2. Interpretability: The ability to pinpoint specific erroneous claims and provide corrections, rather than simply discarding entire agent responses.
3. Practicality: A training-free, plug-and-play solution that balances defense efficacy with computational cost.

The authors conclude that their work advances the understanding of security risks in collaborative LLM systems and catalyzes the development of more rigorous defensive mechanisms. They acknowledge limitations regarding the scope of real-world applications and the inherent confidence calibration issues of LLMs in verifying falsehoods, suggesting future work in incorporating external knowledge for detection.