发现视觉语言模型在操作中会泄漏敏感文本，且防护措施效果有限。
VisualLeakBench: Reproducible Action-Boundary Propagation Failures in Vision-Language Agents
- 构建500张图像的基准测试集，评估视觉文本泄露风险。
- 85.5%的危险文本在渲染后仍会传入工具参数，防御提示仅降低部分风险。
- 适合关注AI安全、隐私泄露与系统防护的研究者和工程师。
视觉语言代理在执行任务前常需解析截图、文档和用户界面内容，随后写入内存、发送消息或调用外部工具。本文研究一种具体失效模式：动作边界传播，即敏感或不安全的可见文本被从图像复制到下游工具参数中。为此提出VisualLeakBench，一个包含500张图像的多样化基准测试集，涵盖界面、聊天、文档、表单和仪表板场景；对其中100张图像的子集，使用四种生产级视觉语言模型（VLM）在两种工作流（笔记捕获与外部移交）下进行评估。基线测试中，个人身份信息（PII）案例的传播率达78.8%，渲染后的不安全文本传播率高达85.5%。采用防御性系统提示后，不安全文本传播率仍达52.6%，而PII传播率降至2.0%，主要通过抑制工具调用实现，而非保留功能实用性。传播率具有工具表面依赖性：搜索类工具能抑制PII传播，但渲染的不安全文本仍可越界。本研究衡量的是从视觉输入到工具参数的传播，而非后续指令执行。此外，提供带标注目标的上界诊断方法，可将多数失败定位在工具边界，但响应端泄露仍为残余风险。
原文摘要 · Abstract (English)
Vision-language agents increasingly consume screenshots, documents, and user interfaces before writing to memory, sending messages, or invoking external tools. We study a concrete failure mode in this setting: action-boundary propagation, where sensitive or unsafe visible text is copied from an image into downstream tool arguments. We present VisualLeakBench, a diversified 500-image benchmark spanning UI, chat, document, form, and dashboard scenes, and evaluate a stratified 100-image agent subset with four production VLM systems under two workflows: note capture and external handoff. At baseline, target strings are propagated into tool arguments in 78.8% of PII cases and 85.5% of rendered unsafe-text cases. Under a defensive system prompt, rendered unsafe-text propagation remains high at 52.6%, while PII tool propagation falls to 2.0%, largely by suppressing tool use rather than preserving utility. Rates are tool-surface dependent: search-like tools suppress PII propagation, but rendered unsafe text still crosses tool boundaries. We measure visual-to-tool propagation rather than downstream instruction execution. We additionally provide a labeled-target oracle upper-bound diagnostic that localizes most failures at the tool boundary while leaving response-side leakage as residual risk.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。