文档类多模态模型在缺乏图像证据时会泄露关联的敏感信息,该研究提出新框架有效防范。
Beyond Visual Evidence: Revealing and Mitigating Relational Privacy Leakage in Document MLLMs

- 设计动态关系遗忘框架,分离并削弱高风险字段关联记忆
- 在噪声数据上测试显示,现有模型隐私泄露率超基准,新方法降低4.8个百分点
- 构建专用评测基准DocPrivacyBench,适合评估文档类MLLM隐私风险
尽管多模态大模型的隐私风险已受关注,但领域特定模型的潜在漏洞仍待深入。本文聚焦身份文档处理中的多模态大语言模型(MLLMs),研究关键信息抽取(KIE)任务中的隐私问题。当输入图像缺乏足够视觉证据时,模型常依赖训练数据中记忆的字段关系推断缺失内容,导致多个相关字段的敏感个人信息泄露。为此,本文提出三项贡献:第一,提出动态关系遗忘框架(DRUF),包含关系解耦遗忘模块与动态集合更新机制,抑制高风险字段对泄漏同时保持KIE性能;第二,构建新评测基准DocPrivacyBench,系统评估模型在视觉证据缺失或极少情况下的隐私脆弱性;第三,基于该基准评估三种MLLMs与六种遗忘方法,分析遗忘后泄漏抑制效果与任务性能保留。结果表明,现有模型在视觉证据不足时普遍存在隐私泄露,尤其在噪声数据上更为显著。相比之下,DRUF相比最强基线提升4.8个百分点的泄漏抑制能力,有效缓解隐私风险并维持强文档信息提取性能。
原文摘要 · Abstract (English)
While the privacy risks of multimodal large language models (MLLMs) have drawn significant attention, the unique vulnerabilities of domain-specific MLLMs remain largely underexplored. Focusing on document understanding MLLMs for identity document processing, this paper investigates the privacy issues inherent in Key Information Extraction (KIE) tasks. We reveal that when input images lack sufficient visual evidence, these models often rely on memorized field relations from training data to infer missing content, thereby leaking multiple correlated fields containing sensitive personal information. To mitigate this risk, we make three key contributions.First, we propose the Dynamic Relational Unlearning Framework (DRUF) which comprises a Relational Decoupling Unlearning (RDU) module and a dynamic set update mechanism. It suppresses the leakage of high-risk field pairs while preserving KIE performance.Second, we introduce DocPrivacyBench, a novel benchmark to systematically evaluate a model's susceptibility to privacy leakage under conditions of absent or minimal visual evidence.Third, we evaluate three MLLMs and six unlearning methods using this benchmark, assessing both post-unlearning leakage suppression and utility preservation.Our results demonstrate that existing MLLMs consistently exhibit privacy leakage when visual evidence is scarce, particularly on noisier datasets. In contrast, DRUF outperforms the strongest baseline by improving leakage suppression by 4.8 percentage points, effectively mitigating privacy risks while maintaining robust document information extraction performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。