通过结构化推理增强安全对齐,有效防御间接提示注入攻击。
ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack
- 引入结构化推理步骤分析用户请求,识别冲突指令
- 在CyberSecEval2上实现94.6%可用性与3.6%攻击成功率
- 适合构建安全可靠的智能代理系统的研究者与开发者
大型语言模型(LLMs)推动了复杂工作流自动化代理系统的兴起,但这些系统极易受到间接提示注入攻击的威胁——恶意指令嵌入外部数据可劫持代理行为。本文提出ReasAlign,一种模型级防御方案,通过引入结构化推理步骤,分析用户查询、检测冲突指令并保持任务连续性以抵御此类攻击。为提升推理逻辑与准确性,设计了一种测试时扩展机制,结合偏好优化的裁判模型对推理轨迹评分并选择最优路径。在多个基准测试中,ReasAlign在保持与未受保护模型相当可用性的同时,显著优于现有最强防护模型Meta SecAlign。在代表性的开放性基准CyberSecEval2上,其可用性达94.6%,攻击成功率仅为3.6%,远超Meta SecAlign的56.4%可用性与74.4%攻击成功率。结果表明,ReasAlign实现了安全与可用性之间的最佳平衡,为现实世界代理系统提供了鲁棒且实用的防御方案。代码与实验结果见https://github.com/leolee99/ReasAlign。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have enabled the development of powerful agentic systems capable of automating complex workflows across various fields. However, these systems are highly vulnerable to indirect prompt injection attacks, where malicious instructions embedded in external data can hijack agent behavior. In this work, we present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks. The core idea of ReasAlign is to incorporate structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks to defend against indirect injection attacks. To further ensure reasoning logic and accuracy, we introduce a test-time scaling mechanism with a preference-optimized judge model that scores reasoning steps and selects the best trajectory. Comprehensive evaluations across various benchmarks show that ReasAlign maintains utility comparable to an undefended model while consistently outperforming Meta SecAlign, the strongest prior guardrail. On the representative open-ended CyberSecEval2 benchmark, which includes multiple prompt-injected tasks, ReasAlign achieves 94.6% utility and only 3.6% ASR, far surpassing the state-of-the-art defensive model of Meta SecAlign (56.4% utility and 74.4% ASR). These results demonstrate that ReasAlign achieves the best trade-off between security and utility, establishing a robust and practical defense against prompt injection attacks in real-world agentic systems. Our code and experimental results could be found at https://github.com/leolee99/ReasAlign.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。