实时检测并阻止大模型多智能体系统中的秘密信息泄露。
PRISM: Generation-Time Detection and Mitigation of Secret Leakage in Multi-Agent LLM Pipelines

- 将泄露风险建模为生成过程中的动态累积问题,逐词评估风险。
- 在2000个任务上实现0%泄露率,精度达1.000,F1达0.832。
- 适合关注AI安全、多智能体系统部署的开发者与研究者。
多智能体大模型系统存在敏感信息通过共享上下文传播并出现在下游输出中的安全风险,即使无恶意意图也会发生。我们将其形式化为传播放大现象:敏感内容在跨智能体传递时泄露风险持续累积。现有防御措施如提示防护、静态模式匹配和大模型判别过滤,均不适用于此场景——或生成后才干预,或依赖表面模式,或引入高延迟且未建模生成过程。为此,我们提出PRISM,一种实时防御机制,将凭据泄露视为生成过程中的序列风险累积问题。在每个解码步骤中,PRISM融合16种信号(涵盖词汇、结构、信息论、行为和上下文特征)生成校准风险评分,支持按词干预(绿黄红三区)。核心观察是:凭据重现前常伴随生成动态的可测量变化,表现为熵坍缩与对数概率集中。结合文本结构线索(如标识符模式检测),这些时间信号可在秘密完全重构前提供早期预警。在覆盖13类攻击、三种压力水平的异构四智能体管道上,2000个任务的对抗基准测试中,PRISM实现F1=0.832,精度=1.000,召回=0.712,任务级泄露率为0.0%,输出实用性保持在0.893。显著优于最强基线Span Tagger(F1=0.719,泄露率15.0%)。
原文摘要 · Abstract (English)
Multi-agent LLM systems introduce a security risk in which sensitive information accessed by one agent can propagate through shared context and reappear in downstream outputs, even without explicit adversarial intent. We formalise this phenomenon as propagation amplification, where leakage risk increases across agent boundaries as sensitive content is repeatedly exposed to downstream generators. Existing defences, including prompt-based safeguards, static pattern matching, and LLM-as-judge filtering, are not designed for this setting: they either operate after generation, rely primarily on surface-form patterns, or add substantial latency without modelling the generation process itself. To resolve these issues, we propose PRISM, a real-time defence that treats credential leakage as a sequential risk accumulation problem during generation. At each decoding step, PRISM combines 16 signals spanning lexical, structural, information-theoretic, behavioural, and contextual features into a calibrated risk score, enabling per-token intervention through green, yellow, and red risk zones. Our central observation is that credential reproduction is often preceded by a measurable shift in generation dynamics, characterised by entropy collapse and increasing logit concentration. When combined with text-structural cues such as identifier-pattern detection, these temporal signals provide an early warning of leakage before a secret is fully reconstructed. Across a 2,000-task adversarial benchmark covering 13 attack categories and three pressure levels in a heterogeneous four-agent pipeline, PRISM achieves F1 = 0.832 with precision = 1.000 and recall = 0.712, while producing no observed leakage on our benchmark (0.0% task-level leak rate) and preserving output utility of 0.893. It substantially outperforms the strongest baseline, Span Tagger, which achieves F1 = 0.719 with a 15.0% task-level leak rate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。