提出信息流管控框架，有效防御RAG中的知识污染攻击。
Cordon-MAS: Defending RAG against Knowledge Poisoning via Information-Flow Control
- 将证据提取、交叉审计与答案生成分隔为权限不同的智能体
- 在五个BEIR数据集上将攻击成功率降低92.4%
- 适合关注大模型安全与可信生成的开发者
检索增强生成（RAG）正广泛应用于高风险场景，但易受类似Confundo的投毒攻击——攻击者优化文档以操纵生成结果。现有防御依赖检测污染证据来阻止危害，但我们发现该假设不成立：模型虽能识别证据矛盾，仍会采纳污染内容。为此提出「隔离原则」——任何具备最终合成能力的代理不得访问未经信任的自然语言证据，并通过CORDON-MAS实现：将证据提取、跨源审计与答案合成拆分为权限不对称的智能体。在五个BEIR数据集上，相比未受保护的RAG，攻击成功率下降92.4%。这将RAG投毒问题从检测转向信息流控制。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) increasingly underpins high-stakes applications, yet remains vulnerable to Confundo-style poisoning where adversarially optimized documents manipulate generated outputs. Existing defenses assume that detecting poisoned evidence prevents harm. We show this assumption is incorrect: models exhibit a monitoring-control gap -- they can detect contradictions in retrieved evidence yet still act on poisoned claims. We introduce the Cordon Principle -- no agent capable of final synthesis may access untrusted natural-language evidence -- and realize it through CORDON-MAS, a compartmentalized framework that enforces this principle architecturally by separating evidence extraction, cross-source audit, and answer synthesis into agents with asymmetric memory privileges. Across five BEIR datasets, CORDON-MAS reduces attack success rate by 92.4\% relative to undefended RAG. This reframes RAG poisoning from a detection problem to an information-flow control problem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。