提出ACE架构,防范大模型应用中的恶意插件攻击。
ACE: A Security Architecture for LLM-Integrated App Systems
- 分两阶段规划:先用可信信息生成抽象计划,再映射为具体执行
- 静态分析确保计划符合安全信息流约束,防御提示注入攻击
- 适合关注大模型应用安全的开发者与系统设计者
LLM集成应用系统通过系统级大模型调用第三方应用,以交错的规划与执行阶段响应用户查询。此类系统引入新攻击面,恶意应用可能破坏规划完整性、执行可用性或引发隐私泄露。本文揭示针对规划完整性,以及执行完整性与可用性的新型攻击,并在近期防御方案IsolateGPT上实现验证。为此提出抽象-具体-执行(ACE)安全架构,通过分离规划阶段:先仅基于可信信息生成抽象计划,再由安装的系统应用映射为具体计划;并通过结构化计划输出的静态分析,验证其满足用户指定的安全信息流约束。执行阶段,ACE在各应用间设置数据与能力隔离屏障,确保执行严格遵循可信抽象计划。实验表明,ACE可抵御InjecAgent和Agent Security Bench中间接提示注入攻击,以及本文新提出的攻击。在LangChain工具使用套件的真实环境评估中,仍保持良好实用性。本架构基于系统安全原则,显著提升大模型系统的抗攻击能力。
原文摘要 · Abstract (English)
LLM-integrated app systems extend the utility of Large Language Models (LLMs) with third-party apps that are invoked by a system LLM using interleaved planning and execution phases to answer user queries. These systems introduce new attack vectors where malicious apps can cause integrity violation of planning or execution, availability breakdown, or privacy compromise during execution. In this work, we identify new attacks impacting the integrity of planning, as well as the integrity and availability of execution in LLM-integrated apps, and demonstrate them against IsolateGPT, a recent solution designed to mitigate attacks from malicious apps. We propose Abstract-Concrete-Execute (ACE), a new secure architecture for LLM-integrated app systems that provides security guarantees for system planning and execution. Specifically, ACE decouples planning into two phases by first creating an abstract execution plan using only trusted information, and then mapping the abstract plan to a concrete plan using installed system apps. We verify that the plans generated by our system satisfy user-specified secure information flow constraints via static analysis on the structured plan output. During execution, ACE enforces data and capability barriers between apps, and ensures that the execution is conducted according to the trusted abstract plan. We show experimentally that ACE is secure against attacks from the InjecAgent and Agent Security Bench benchmarks for indirect prompt injection, and our newly introduced attacks. We also evaluate the utility of ACE in realistic environments, using the Tool Usage suite from the LangChain benchmark. Our architecture represents a significant advancement towards hardening LLM-based systems using system security principles.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。