提出新方法在无保留图像时遗忘多模态模型中的人脸隐私信息。
AIM: Anchor Identity Features, Then Match for Multimodal Large Language Model Unlearning
- 用通用视觉提示锚定遗忘目标，通过费舍尔约束匹配视觉编码器。
- 遗忘目标人物信息同时保持其他身份与视觉理解能力。
- 适用于需删除个人数据但无法获取原始图像的隐私保护场景。
多模态大语言模型（MLLM）在微调数据中可能记忆特定人物的身份信息，当该人物请求删除时会带来隐私风险。现有遗忘方法通常假设删除时可访问保留图像或真实答案，这在实际中不现实。本文研究在无法获取保留图像时的身份遗忘问题。分析发现，身份相关与视觉感知类问题在微调后的隐藏状态中占据不同区域，且组织方式不同：身份问题按人聚类，感知问题按问题类型聚类。这表明可抑制身份知识而不影响通用视觉感知。基于此，我们提出两阶段方法AIM，先用通用视觉提示锚定遗忘目标，再在费舍尔约束下匹配视觉编码器。大量实验表明，AIM在实现良好身份遗忘的同时，保留了非删除身份、先前知识及相同图像上的视觉感知能力。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) can memorize identity-specific facts about people in their fine-tuning data, creating privacy risks when a person requests deletion. Existing MLLM unlearning methods often assume access to retain images or ground-truth answers during deletion, which is unrealistic in many practical scenarios. We study identity unlearning when retain images are unavailable at deletion time. Our analysis shows that identity and visual-perception questions occupy distinct regions in fine-tuned hidden states and are organized differently: identity questions cluster by person, whereas perception questions cluster by question type. This suggests that identity knowledge can be suppressed without erasing general visual perception. Building on this observation, we propose AIM, a two-stage method that anchors an identity-forgetting target with a universal visual prompt and then matches the vision encoder to that target under a Fisher-based constraint. Extensive experiments show that AIM achieves competitive identity forgetting while preserving non-deleted identities, prior knowledge, and visual perception on the same images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。