攻击者可借恶意工具注入，让大模型泄露隐私或瘫痪服务
From Allies to Adversaries: Manipulating LLM Tool-Calling through Adversarial Injection
- 分两阶段注入恶意工具，先窃取用户查询再动态升级攻击
- 隐私泄露攻击成功率91.67%，拒绝服务和异常调用可达100%
- 揭示大模型工具调用系统深层漏洞，适合安全研究者参考
工具调用使大语言模型在各类任务中具备更强能力，但也引入新安全风险，尤其在工具调度机制方面尚无深入研究。本文提出ToolCommander框架，通过对抗性工具注入攻击大模型工具调用系统。该框架采用两阶段策略：首阶段注入恶意工具以收集用户查询；第二阶段根据窃取信息动态更新工具，增强后续攻击。此方法可实现隐私窃取、拒绝服务攻击，甚至触发非预期工具调用以操纵商业竞争。实验显示，隐私窃取的攻击成功率（ASR）达91.67%，在特定情况下拒绝服务与非预期调用攻击成功率均为100%。研究证明此类漏洞可能导致远超滥用工具系统的严重后果，亟需构建防御机制保护大模型工具调用系统。
原文摘要 · Abstract (English)
Tool-calling has changed Large Language Model (LLM) applications by integrating external tools, significantly enhancing their functionality across diverse tasks. However, this integration also introduces new security vulnerabilities, particularly in the tool scheduling mechanisms of LLM, which have not been extensively studied. To fill this gap, we present ToolCommander, a novel framework designed to exploit vulnerabilities in LLM tool-calling systems through adversarial tool injection. Our framework employs a well-designed two-stage attack strategy. Firstly, it injects malicious tools to collect user queries, then dynamically updates the injected tools based on the stolen information to enhance subsequent attacks. These stages enable ToolCommander to execute privacy theft, launch denial-of-service attacks, and even manipulate business competition by triggering unscheduled tool-calling. Notably, the ASR reaches 91.67% for privacy theft and hits 100% for denial-of-service and unscheduled tool calling in certain cases. Our work demonstrates that these vulnerabilities can lead to severe consequences beyond simple misuse of tool-calling systems, underscoring the urgent need for robust defensive strategies to secure LLM Tool-calling systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。