提出新方法实现精准删除隐私信息,同时不损伤模型看图能力。
Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning
- 通过限定遗忘范围,只删敏感记忆,保留其他能力。
- 在基准测试中,现有方法均破坏模型看图能力。
- 适合关注模型隐私安全与性能平衡的研究者使用。
多模态大语言模型在训练中可能无意记住敏感信息。现有去记忆方法虽能移除这些内容,但常严重损害模型的基础能力,如通用图像理解。为解决这一问题,我们提出良性记忆遗忘,即精准删除特定隐私知识的同时严格保留无关能力。为此,我们构建了首个联合量化评估去记忆效果与图像理解保留程度的基准S-MLLMUn Bench。同时提出雕刻式记忆遗忘适配器(SMFA),通过先微调模型将敏感输出替换为拒绝响应,生成遗忘适配器,再利用保持锚点引导的掩码机制保护无关知识。大量实验表明,现有方法无法实现良性遗忘,而SMFA作为有效基线,成功实现目标知识清除且不损害模型基础视觉能力。
原文摘要 · Abstract (English)
Multimodal large language models (MLLMs) can inadvertently memorize privacy-sensitive information during training. While existing unlearning methods can remove such content, they often severely degrade the model's foundational capabilities, such as general image understanding. This critical shortfall motivates our investigation into benign memory forgetting, the precise removal of targeted, privacy-sensitive knowledge while rigorously preserving unrelated capabilities. To pioneer and evaluate progress toward this objective, we introduce S-MLLMUn Bench, the first benchmark designed to jointly and quantitatively assess an unlearning method's efficacy in knowledge erasure and the preservation of image understanding. Furthermore, we propose the Sculpted Memory Forgetting Adapter (SMFA), a new framework that enables benign memory forgetting. SMFA confines forgetting to designated memory regions, maintaining overall model performance. By initially fine-tuning the model to replace sensitive outputs with refusals, SMFA generates a memory forgetting adapter, followed by a retaining anchor-guided masking mechanism that safeguards unrelated knowledge. Extensive experiments on S-MLLMUn Bench demonstrate that existing methods fail to achieve benign forgetting, whereas our proposed SMFA serves as an effective baseline, successfully achieving targeted knowledge erasure without compromising the model's foundational visual capabilities. Code and data are available at https://github.com/zeng-zhen/S-MLLMUn.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。