多智能体系统内部通信存在隐私泄露，输出审计无法发现。
AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems
- 构建七类通信路径的隐私泄露基准，检测内部通道风险
- 内部消息泄露率达68.8%，远超最终输出的27.2%
- 适合关注多智能体系统安全的开发者与研究者
多智能体大语言模型系统带来隐私风险，现有仅评估输出的基准无法衡量。当智能体协作时，敏感数据可能通过智能体间消息、共享内存和工具参数等内部通道传递，这些路径通常不在最终输出审计范围内。我们提出AgentLeak，用于评估多智能体系统中的内部通道隐私泄露。该基准针对七种隐私相关通信路径进行测试，覆盖1,000个场景，涉及医疗、金融、法律和企业领域，使用五款生产级模型（GPT-4o、GPT-4o-mini、Claude 3.5 Sonnet、Mistral Large、Llama 3.3 70B）和4,979条验证执行轨迹。结果表明，多智能体配置虽降低最终输出泄露（C1：27.2% vs 单智能体43.2%），但引入内部通道使系统总暴露率升至68.9%（涵盖C1、C2、C5）。其中，智能体间消息泄露（C2）达68.8%，远高于最终输出的27.2%，意味着输出审计遗漏41.7%的违规行为。在所有模型与四类领域中，始终呈现C2 ≥ C1的趋势。这表明，在所考察的协调-工作模式下，多智能体系统的隐私风险主要由架构性协调通道决定，而非仅由最终输出行为体现——其风险来自常规输出防御所无法触及的内部通道。
原文摘要 · Abstract (English)
Multi-agent Large Language Model (LLM) systems create privacy risks that current output-only benchmarks cannot measure. When agents coordinate on tasks, sensitive data may pass through inter-agent messages, shared memory, and tool arguments, all pathways that final-output audits typically do not inspect. We introduce AgentLeak, a benchmark for evaluating internal-channel privacy leakage in multi-agent LLM systems. AgentLeak instruments seven privacy-relevant communication pathways and provides a large-scale empirical evaluation focused on final outputs, inter-agent messages, and shared memory. Across 1,000 scenarios spanning healthcare, finance, legal, and corporate domains, five production LLMs (GPT-4o, GPT-4o-mini, Claude 3.5 Sonnet, Mistral Large, and Llama 3.3 70B), and 4,979 validated execution traces, we find that multi-agent configurations reduce final-output leakage (C1: 27.2% vs 43.2% in single-agent mode) compared with single-agent baselines but introduce internal channels that raise total system exposure to 68.9% (aggregated across C1, C2, C5). Inter-agent messages (C2) leak at 68.8%, compared with 27.2% for final outputs (C1), meaning that output-only audits miss 41.7% of violations. Across all five models and four domains, the pattern C2 $\geq$ C1 holds consistently. These results suggest, within the evaluated coordinator-worker setting, that privacy risk in multi-agent systems is strongly shaped by architectural coordination channels rather than final-output behavior alone: it arises from internal channels that remain invisible to standard output-level defenses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。