让多模态模型安全删知识，不破坏理解能力。
SineProject: Machine Unlearning for Stable Vision Language Alignment
- 用正弦调制参数增强冻结的投影层，稳定跨模态对齐。
- 在安全与隐私删知识任务中，完全遗忘目标信息且拒答正常请求减少70%以上。
- 适合需要精准删知识又不想丢理解力的AI应用开发者。
多模态大语言模型（MLLMs）需在不重新训练的情况下遗忘特定知识（如敏感或危险信息）。现有方法常破坏视觉-语言对齐，导致模型拒绝所有查询。我们发现根源在于未更新的投影层，其雅可比矩阵严重病态，引发优化不稳定和跨模态嵌入偏移。提出SineProject：在冻结投影层中加入正弦调制的可训练参数，提升雅可比谱条件数，稳定对齐过程。在LLaVA v1.5 7B与13B的主流安全与隐私删知识基准上，该方法显著降低正常查询拒答率，同时实现目标信息完全遗忘，达成当前最优遗忘-保留权衡，计算开销可忽略。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) increasingly need to forget specific knowledge such as unsafe or private information without requiring full retraining. However, existing unlearning methods often disrupt vision language alignment, causing models to reject both harmful and benign queries. We trace this failure to the projector network during unlearning, its Jacobian becomes severely illconditioned, leading to unstable optimization and drift in cross modal embeddings. We introduce SineProject, a simple method that augments the frozen projector with sinusoidally modulated trainable parameters, improving the Jacobian's spectral conditioning and stabilizing alignment throughout unlearning. Across standard safety and privacy unlearning benchmarks using LLaVA v1.5 7B and 13B, SineProject reduces benign query refusals while achieving complete forgetting of targeted information, yielding state of the art forget retain trade offs with negligible computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。