无需训练和数据，即可精准删除CLIP模型中特定物体知识。
Selective, Controlled and Domain-Agnostic Unlearning in Pretrained CLIP: A Training- and Data-Free Approach
- 利用文本提示与合成视觉原型构建多模态空空间，实现无损遗忘。
- 支持全局、领域特异、选择性三种遗忘模式，灵活可控。
- 适用于需隐私保护或知识更新的AI应用，如内容过滤。
预训练模型如CLIP在自然图像、艺术渲染和抽象表示等多样视觉领域展现出出色的零样本分类能力。然而，实际应用常需移除特定物体类别信息，而不依赖额外数据或重新训练，且不损害模型在无关任务上的性能。本文提出一种全新的训练-数据自由遗忘框架，支持三种不同遗忘范式：(1) 在所有领域中全局遗忘选定物体；(2) 领域特异性知识移除（例如消除素描表征但保留照片识别）；(3) 在特定领域内完全遗忘。通过协同整合文本提示与从CLIP联合嵌入空间生成的合成视觉原型，构建多模态空空间，该方法高效去除目标类别信息，同时保留其余知识。相比现有基于重训练的方法，本方案克服了局限性，提供了一种灵活且计算高效的可控遗忘解决方案。
原文摘要 · Abstract (English)
Pretrained models like CLIP have demonstrated impressive zero-shot classification capabilities across diverse visual domains, spanning natural images, artistic renderings, and abstract representations. However, real-world applications often demand the removal (or "unlearning") of specific object classes without requiring additional data or retraining, or affecting the model's performance on unrelated tasks. In this paper, we propose a novel training- and data-free unlearning framework that enables three distinct forgetting paradigms: (1) global unlearning of selected objects across all domains, (2) domain-specific knowledge removal (e.g., eliminating sketch representations while preserving photo recognition), and (3) complete unlearning in selective domains. By leveraging a multimodal nullspace through synergistic integration of text prompts and synthesized visual prototypes derived from CLIP's joint embedding space, our method efficiently removes undesired class information while preserving the remaining knowledge. This approach overcomes the limitations of existing retraining-based methods and offers a flexible and computationally efficient solution for controlled model forgetting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。