动态劫持大模型推理,让智能代理执行恶意操作
UDora: A Unified Red Teaming Framework against LLM Agents by Dynamically Hijacking Their Own Reasoning
- 通过分析代理推理过程,精准插入干扰扰动
- 在三个数据集上均显著优于现有攻击方法
- 适合安全研究者测试大模型抗攻击能力
具备外部工具的大语言模型代理在网页购物、自动邮件回复和金融交易等复杂任务中日益强大,但这也放大了对抗攻击的风险,尤其当代理可访问敏感外部功能时。然而,操控代理执行特定恶意行为或调用特定工具仍具挑战性,因其在执行最终动作前会进行大量推理与规划。本文提出UDora,一种统一的红队框架,可动态劫持代理的推理过程以诱导恶意行为。具体而言,UDora首先生成模型对给定任务的推理轨迹,随后自动识别最优插入点并注入目标扰动,将扰动后的推理作为优化的代理响应。通过迭代此过程,可诱导代理执行指定恶意操作或调用特定恶意工具。该方法在三个LLM代理数据集上均表现出优于现有方法的攻击效果。代码已开源:https://github.com/AI-secure/UDora。
原文摘要 · Abstract (English)
Large Language Model (LLM) agents equipped with external tools have become increasingly powerful for complex tasks such as web shopping, automated email replies, and financial trading. However, these advancements amplify the risks of adversarial attacks, especially when agents can access sensitive external functionalities. Nevertheless, manipulating LLM agents into performing targeted malicious actions or invoking specific tools remains challenging, as these agents extensively reason or plan before executing final actions. In this work, we present UDora, a unified red teaming framework designed for LLM agents that dynamically hijacks the agent's reasoning processes to compel malicious behavior. Specifically, UDora first generates the model's reasoning trace for the given task, then automatically identifies optimal points within this trace to insert targeted perturbations. The resulting perturbed reasoning is then used as a surrogate response for optimization. By iteratively applying this process, the LLM agent will then be induced to undertake designated malicious actions or to invoke specific malicious tools. Our approach demonstrates superior effectiveness compared to existing methods across three LLM agent datasets. The code is available at https://github.com/AI-secure/UDora.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。