arXiv:2510.04217cs.LGcs.AI2025-10被引 7

无需训练即可在测试时擦除多模态大模型记忆,高效且不损伤可用知识。

MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering

  • 通过激活操控实现测试时遗忘,不更新参数,计算开销小。
  • 在LLaVA-1.5和Qwen-2.5-VL上实现更强遗忘效果,性能优于现有方法。
  • 输入感知机制智能判断何时干预,避免误删有用信息,适合部署场景。

多模态大语言模型(MLLMs)在视觉-语言任务中表现卓越,但其大规模应用引发对记忆隐私数据、过时知识和有害内容的担忧。现有遗忘方法多基于训练阶段策略(如梯度上升或偏好优化),存在计算成本高、不可逆且易破坏保留知识的问题。本文提出MLLMEraser,一种输入感知的、无需训练的测试时遗忘框架。该方法通过对比对抗扰动的图文回忆对与遗忘对,构建多模态擦除方向,捕捉文本与视觉差异。为避免不必要的干扰,设计输入感知的激活操控机制,动态决定是否及如何应用擦除方向,从而在保留原有知识效用的同时,精准擦除指定内容。在LLaVA-1.5和Qwen-2.5-VL上的实验表明,MLLMEraser持续优于当前最优基线,实现更强遗忘性能,计算成本更低,且对模型可用性影响极小。

原文摘要 · Abstract (English)

Multimodal large language models (MLLMs) have demonstrated remarkable capabilities across vision-language tasks, yet their large-scale deployment raises pressing concerns about memorized private data, outdated knowledge, and harmful content. Existing unlearning approaches for MLLMs typically adapt training-based strategies such as gradient ascent or preference optimization, but these methods are computationally expensive, irreversible, and often distort retained knowledge. In this work, we propose MLLMEraser, an input-aware, training-free framework for test-time unlearning. Our approach leverages activation steering to enable dynamic knowledge erasure without parameter updates. Specifically, we construct a multimodal erasure direction by contrasting adversarially perturbed, knowledge-recall image-text pairs with knowledge-erasure counterparts, capturing both textual and visual discrepancies. To prevent unnecessary interference, we further design an input-aware steering mechanism that adaptively determines when and how the erasure direction should be applied, preserving utility on retained knowledge while enforcing forgetting on designated content. Experiments on LLaVA-1.5 and Qwen-2.5-VL demonstrate that MLLMEraser consistently outperforms state-of-the-art MLLM unlearning baselines, achieving stronger forgetting performance with lower computational cost and minimal utility degradation.

多模态遗忘测试时激活操控

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。