用任务对齐防御大模型代理的间接提示注入攻击
The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents
- 通过验证每步操作是否服务于用户目标来防御攻击
- 在GPT-4o上将攻击成功率降至2.07%，任务完成率仍达69.79%
- 适合需要安全执行复杂任务的智能代理系统
大型语言模型（LLM）代理正被广泛用于具备工具集成能力的对话助手，可完成复杂的现实任务。然而，其与外部系统交互的能力也带来了显著安全风险，尤其面临间接提示注入攻击：恶意指令嵌入外部数据源，可使代理偏离用户意图。现有基于规则约束、来源聚焦和认证协议的防御手段虽有一定效果，但难以兼顾安全性与任务功能。本文提出一种新视角，将代理安全定义为任务对齐——确保每个动作均服务于用户目标。基于此，我们设计了测试时防御机制Task Shield，系统性验证每条指令和工具调用是否有助于达成用户指定目标。在AgentDojo基准上的实验表明，Task Shield将攻击成功率降低至2.07%，同时在GPT-4o上保持69.79%的任务实用性。
原文摘要 · Abstract (English)
Large Language Model (LLM) agents are increasingly being deployed as conversational assistants capable of performing complex real-world tasks through tool integration. This enhanced ability to interact with external systems and process various data sources, while powerful, introduces significant security vulnerabilities. In particular, indirect prompt injection attacks pose a critical threat, where malicious instructions embedded within external data sources can manipulate agents to deviate from user intentions. While existing defenses based on rule constraints, source spotlighting, and authentication protocols show promise, they struggle to maintain robust security while preserving task functionality. We propose a novel and orthogonal perspective that reframes agent security from preventing harmful actions to ensuring task alignment, requiring every agent action to serve user objectives. Based on this insight, we develop Task Shield, a test-time defense mechanism that systematically verifies whether each instruction and tool call contributes to user-specified goals. Through experiments on the AgentDojo benchmark, we demonstrate that Task Shield reduces attack success rates (2.07\%) while maintaining high task utility (69.79\%) on GPT-4o.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。