提出隐私恢复评估新方法，解决多模态大模型编辑中的隐私泄露盲区。
When Privacy Meets Recovery: The Overlooked Half of Surrogate-Driven Privacy Preservation for MLLM Editing
- 构建SPPE数据集，包含多种隐私类型与用户指令。
- 在多个任务上实现隐私恢复与编辑质量的平衡，提升可用性。
- 适用于关注隐私安全的AI系统开发与评测人员。
多模态大语言模型（MLLM）中的隐私泄露问题长期难以解决。现有研究虽能有效隐藏私密信息，却常忽略对用户隐私真实性及恢复质量的评估。本文首次聚焦于在不同MLLM场景下恢复代理保护数据的关键挑战。我们通过构建涵盖广泛隐私类别和用户指令的SPPE（Surrogate Privacy Protected Editable）数据集，提供受保护的代理数据及其对应的MLLM编辑版本，从而可直接评估隐私恢复质量。将隐私恢复建模为基于互补多模态信号的引导生成任务，提出统一方法，在保持MLLM生成编辑保真度的同时可靠重建私密内容。在SPPE和InstructPix2Pix上的实验表明，该方法在多样视觉内容与编辑任务中具有良好泛化能力，实现了隐私保护与模型可用性的良好平衡。
原文摘要 · Abstract (English)
Privacy leakage in Multimodal Large Language Models (MLLMs) has long been an intractable problem. Existing studies, though effectively obscure private information in MLLMs, often overlook the evaluation of the authenticity and recovery quality of user privacy. To this end, this work uniquely focuses on the critical challenge of how to restore surrogate-driven protected data in diverse MLLM scenarios. We first bridge this research gap by contributing the SPPE (Surrogate Privacy Protected Editable) dataset, which includes a wide range of privacy categories and user instructions to simulate real MLLM applications. This dataset offers protected surrogates alongside their various MLLM-edited versions, thus enabling the direct assessment of privacy recovery quality. By formulating privacy recovery as a guided generation task conditioned on complementary multimodal signals, we further introduce a unified approach that reliably reconstructs private content while preserving the fidelity of MLLM-generated edits. The experiments on both SPPE and InstructPix2Pix further show that our approach generalizes well across diverse visual content and editing tasks, achieving a strong balance between privacy protection and MLLM usability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。