针对企业文档设计的抗提示注入新方法,兼顾安全与实用。
PARSE: Provenance-Aware Retrieval Sanitization for Professional Domain LLM Agents
- 基于文档出处感知的分步净化流程,识别注入风险句并保留事实
- 在真实企业文档上将攻击成功率降至15.6%,比基线降低38%
- 适合金融、法律、医疗等专业领域大模型应用开发者使用
在合成基准上表现优异的提示注入防御方法,在真实企业文档中失效。我们构建了一个包含122个任务的实文档基准,覆盖金融、法律、医疗、科学和DevOps五个专业领域,使用实际的SEC文件、联邦公报规则、PubMed摘要、arXiv论文和GitHub故障复盘。重写(paraphrasing)在合成数据上最强,但在真实文档上未显著降低攻击成功率(p=0.500),且实用性从91.8%降至82.8%。我们提出PARSE(Provenance-Aware Retrieval Sanitization),一种领域感知、事实保全的净化流水线:按注入概率分类句子,提取结构化事实后重写,并通过一致性检查循环验证。直接性门控将59%的文档引导至轻量路径,集中计算资源于高风险文档。PARSE实现15.6%的攻击成功率,较25.4%基线降低38%,同时保持86.9%的实用性,该结果统计显著(p=0.014,充分幂次),是唯一兼具显著性与高实用性的方案。建议从业者在真实领域文档上评估防御,而非依赖合成代理。
原文摘要 · Abstract (English)
Prompt injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents, which are longer, denser, and interleave legitimate authority language with factual content. We demonstrate this gap with a real-document benchmark of 122 tasks across five professional domains (financial, legal, medical, scientific, DevOps) using actual SEC filings, Federal Register rules, PubMed abstracts, arXiv papers, and GitHub postmortems. Paraphrasing, the strongest defense on synthetic benchmarks, shows no statistically significant attack success rate reduction on real documents (p=0.500) while degrading utility from 91.8% to 82.8%. We introduce PARSE (Provenance-Aware Retrieval Sanitization), a domain-aware, fact-preserving sanitization pipeline that classifies each sentence by injection likelihood, extracts structured facts before rewriting, and verifies fact preservation via a consistency-checking loop. A directiveness gate routes 59% of real enterprise documents to a lightweight path, concentrating computational cost on high-risk documents. PARSE achieves 15.6% attack success rate -- a 38% reduction versus the 25.4% baseline -- at 86.9% utility, the only condition that is both statistically significant (p=0.014, adequately powered) and maintains near-baseline utility. Practitioners should evaluate defenses on domain-matched real documents, not synthetic proxies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。