揭露LLM多智能体开发系统隐蔽漏洞,提出攻防方案
Shadows in the Code: Exploring the Risks and Defenses of LLM-based Multi-Agent Software Development Systems
- 设计隐性恶意行为注入攻击,操纵智能体生成带后门的代码
- 攻击成功率最高达93%,编码与测试阶段智能体最易被劫持
- 提出防御机制并显著降低攻击成功率,适合安全研究人员参考
基于大语言模型的多智能体软件开发系统虽降低了技术门槛,但引入了未充分探索的安全风险。本文识别出两类高危场景:恶意用户使用良性智能体(MU-BA)与良性用户遭遇恶意智能体(BU-MA)。提出隐性恶意行为注入攻击(IMBIA),证明可使系统生成表面无害实则含恶意功能的代码。在ChatDev、MetaGPT和AgentVerse框架中评估,IMBIA在MU-BA场景下成功率达93%、45%、71%,在BU-MA场景下为71%、84%、45%。所提防御机制Adv-IMBIA显著降低攻击成功率,尤其在MU-BA场景效果明显。分析发现,编码与测试阶段的智能体风险更高,需重点防护。研究揭示多智能体开发系统亟需强化安全措施,并提供针对性、高效资源利用的防御指南。
原文摘要 · Abstract (English)
The rapid advancement of Large Language Model (LLM)-driven multi-agent systems has significantly streamlined software developing tasks, enabling users with little technical expertise to develop executable applications. While these systems democratize software creation through natural language requirements, they introduce significant security risks that remain largely unexplored. We identify two risky scenarios: Malicious User with Benign Agents (MU-BA) and Benign User with Malicious Agents (BU-MA). We introduce the Implicit Malicious Behavior Injection Attack (IMBIA), demonstrating how multi-agent systems can be manipulated to generate software with concealed malicious capabilities beneath seemingly benign applications, and propose Adv-IMBIA as a defense mechanism. Evaluations across ChatDev, MetaGPT, and AgentVerse frameworks reveal varying vulnerability patterns, with IMBIA achieving attack success rates of 93%, 45%, and 71% in MU-BA scenarios, and 71%, 84%, and 45% in BU-MA scenarios. Our defense mechanism reduced attack success rates significantly, particularly in the MU-BA scenario. Further analysis reveals that compromised agents in the coding and testing phases pose significantly greater security risks, while also identifying critical agents that require protection against malicious user exploitation. Our findings highlight the urgent need for robust security measures in multi-agent software development systems and provide practical guidelines for implementing targeted, resource-efficient defensive strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。