不依赖保留集，精准抹除模型中特定人物信息
Where Identity Lives: Localized, Retain-Free Identity Unlearning in Multimodal Large Language Models
- 定位身份信息存储层，仅在早期到中期解码器MLP层更新
- 在忘记集上训练时保持图像-语言对齐能力，性能优于同类方法
- 适合需隐私保护的多模态大模型后期编辑场景
在多模态大模型部署后，移除特定个体的信息至关重要。现有方法依赖难以获取的保留集，重建该集会重新引发隐私风险。仅从遗忘集进行遗忘会损害共享的视觉-语言计算，影响感知能力。本文将无保留集遗忘视为定位问题：因果追踪、权重移植和费雪重叠均指向早期至中期解码器MLP层是身份信息的存储位置，且这些层的修改不会显著破坏视觉能力。基于此，提出路径感知视觉属性锚定（PAVA）方法，将更新限制在这些层，并结合遗忘损失与视觉属性锚定，通过仅从遗忘图像中提炼模型预遗忘阶段的答案来保留图像相关行为。在MLLMU-Bench和ReMem数据集上，PAVA在仅使用遗忘集的方法中实现了最优的遗忘-保留权衡，且性能可媲美基于保留集的基线方法。
原文摘要 · Abstract (English)
Removing a specific individual's information from multimodal large language models (MLLMs) is often needed after deployment, but existing methods rely on a retain set, which is hardest to obtain at that point, and rebuilding it recreates the privacy exposure that unlearning aims to remove. Forgetting from the forget set alone instead damages the shared visual-language computation, harming perception. We cast retain-free unlearning as a localization problem: causal tracing, weight transplant, and Fisher overlap all point to early-to-mid decoder MLPs as the layers where identity information is stored and, unlike other module families, can be modified without substantially disrupting vision. We turn this into Pathway-Aware Visual-attribute Anchoring (PAVA), which confines updates to these layers and pairs a forget loss with a visual-attribute anchor that preserves image-grounded behavior by distilling the model's own pre-unlearning answers from the forget images alone. On MLLMU-Bench and ReMem, PAVA gives the strongest forget-retain trade-off among forget-set-only methods and remains competitive with retain-based baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。