arXiv:2607.18847cs.CRcs.AI2026-07

提前加固智能体应用,防止数据泄露和工具滥用。

Data Leakage Prevention in Agentic Applications via Preemptive Hardening

论文配图:Data Leakage Prevention in Agentic Applications via Preemptive Hardening
图 1 · 摘自论文原文
  • 部署前扫描代码与提示模板,自动识别泄漏风险并生成修复补丁。
  • 修复后通过恶意攻击和正常输入测试,确保功能不受影响。
  • 无需运行时监控,可彻底消除基础越狱攻击导致的泄露。

智能体系统将大语言模型驱动的规划与外部工具接口结合,易因指令/数据边界失效或提示注入攻击引发数据泄露和工具滥用。在跨多个代码库和异构智能体的工作流中,持续实施必要控制尤为困难。为此,我们提出一种部署前的扫描、加固与验证流水线。该流水线分析提示模板、工具接口及调用代码,识别导致泄漏的模式,并生成可操作的修复方案。加固阶段优先处理高风险工具,采用最小侵入式策略,包括模式收紧、边界净化、白名单工具管控及最小权限检查。验证阶段自动生成模拟越狱、指令覆盖和工具操控的攻击输入,以及良性任务变体,以确认修复后应用功能仍正常。我们在五个真实世界智能体应用及AgentDojo基准上评估该方法,均发现重复的泄漏模式,并生成可无缝集成的修复代码。修复后,在基础越狱和指令覆盖攻击下实现100%泄漏消除,在压力诱导操纵条件下泄漏减少91%,且无需持续运行时策略执行。

原文摘要 · Abstract (English)

Agentic systems integrate LLM driven planning with interfaces to external tools, making data leakage and tool misuse feasible via instruction/data boundary failures and prompt injection attacks. Enforcing required controls consistently is particularly challenging in workflows spanning many codebases and heterogeneous agents. To address this challenge in multi agentic systems, we present a pre-deployment pipeline for scanning, hardening, and validation of agentic applications. The pipeline analyzes prompt templates, tool interfaces, and tool-invocation code to identify leakage-enabling patterns and generate actionable patches. The hardened application is then validated through adversarial prompt injection attacks and benign input variations ensuring that mitigations do not disrupt intended behavior. In the hardening stage, high-risk tools are prioritized, and minimally invasive mitigations are applied, including schema tightening, boundary sanitization, allowlist-based tool gating, and least-privilege checks. In the validation stage, the pipeline automatically generates attack inputs that mimic jailbreaks, instruction overrides, and tool-targeted manipulation, along with benign task variants, to confirm that the functionality of the hardened application is preserved after remediation. We evaluated the pipeline on five real-world agentic applications, as well as on the AgentDojo benchmark. Across all applications, the proposed pipeline identified recurring leakage-enabling patterns and generated patches that can be integrated without disrupting the intended application behavior. The resulting modifications of application code were shown to eliminate leaks when targeted by basic jailbreak and instruction-override attacks, achieving a 100% reduction in leakage, and reduce leaks by 91% under conditions of stress-induced manipulation, without the need of continuous runtime policy enforcement.

智能体安全数据泄露提示注入代码加固

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。