提出新基准PPE-Bench,评估多模态模型在隐私与公共信息纠缠下的遗忘能力
PPE-Bench: A Benchmark for Evaluating MLLM Unlearning under Private-Public Entanglement

- 构建含私有与公共信息纠缠的图像数据集用于评测
- 现有方法虽能减少隐私泄露但常损害公共信息
- 适用于关注模型隐私保护与知识平衡的研究者
多模态大语言模型(MLLM)虽具备强大能力,但可能从网络数据中记忆私人信息,引发隐私担忧。机器遗忘提供了一种无需从头训练即可移除此类知识的方法。然而,现有MLLM遗忘基准存在两大局限:其一,依赖仅包含单一目标个体的简化图像,无法反映真实照片的视觉复杂性;其二,通常假设遗忘集与保留集完全分离,忽略了私人信息常与良性公共信息视觉纠缠的事实。例如,某私人个体可能与公众人物同框或出现在著名地标前,此时遗忘私人目标不应破坏公共上下文。为解决上述问题,我们提出PPE-Bench,一个在隐私-公共纠缠条件下评估MLLM遗忘能力的新基准。每张图像均包含需遗忘的目标个体及需保留的公共信息(如公众人物、地标)。我们进一步引入两种简单而有效的策略以更好保留公共信息。实验表明,现有遗忘方法虽可降低隐私泄露,但常显著损害邻近公共信息。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) have shown strong capabilities, but they may memorize private information from web data, raising privacy concerns. Machine unlearning offers a way to remove such private knowledge without retraining from scratch. However, existing MLLM unlearning benchmarks have two major limitations. First, they rely on simplified images that contain only the single target individual, failing to reflect the visual complexity of real-world photos. Second, they typically assume that the forget set and retain set are fully separated, ignoring the fact that private information is often visually entangled with benign public information. For example, a private individual may appear with a public figure or in front of a well-known landmark, where unlearning the private target should not damage the public context. To address these limitations, we propose PPE-Bench, a new benchmark for evaluating MLLM unlearning under private-public entanglement. Each image contains a target individual to be forgotten and public information to be preserved, including public figure and landmark. We further introduce two simple but effective methods to better preserve public information during unlearning. Through experiments, we find that existing unlearning methods can reduce private information leakage, but often substantially harm adjacent public information.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。