用CLIP引导的超网络实现动态可控的模型概念删除。
UnHype: CLIP-Guided Hypernetworks for Dynamic LoRA Unlearning
- 用超网络动态生成LoRA权重,根据输入语义自适应调整。
- 在物体、名人和敏感内容删除任务中均有效,支持多概念并行擦除。
- 适合需要精准控制生成内容的场景,如隐私保护与内容安全。
大规模扩散模型的兴起引发了对其潜在滥用的担忧,尤其是生成真实但有害或社会性破坏内容的风险。这推动了机器可撤销学习的发展,即在不损害模型整体生成能力的前提下,选择性移除特定知识或概念。低秩适配(LoRA)已成为一种高效的目标化撤销学习方法,但现有方法在概念语义适应性上有限,难以平衡相关概念的消除与更广泛意义的泛化能力,且多概念同时擦除时存在可扩展性问题。为此,我们提出UnHype框架,将超网络引入单概念和多概念LoRA训练中。该架构可直接嵌入Stable Diffusion及现代基于流的文本到图像模型,表现出稳定的训练行为和有效的概念控制。推理时,超网络根据CLIP嵌入动态生成适配的LoRA权重,实现更上下文感知的可扩展撤销学习。我们在物体擦除、名人擦除和敏感内容移除等挑战性任务中评估了UnHype,验证了其有效性与通用性。
原文摘要 · Abstract (English)
Recent advances in large-scale diffusion models have intensified concerns about their potential misuse, particularly in generating realistic yet harmful or socially disruptive content. This challenge has spurred growing interest in effective machine unlearning, the process of selectively removing specific knowledge or concepts from a model without compromising its overall generative capabilities. Among various approaches, Low-Rank Adaptation (LoRA) has emerged as an effective and efficient method for fine-tuning models toward targeted unlearning. However, LoRA-based methods often exhibit limited adaptability to concept semantics and struggle to balance removing closely related concepts with maintaining generalization across broader meanings. Moreover, these methods face scalability challenges when multiple concepts must be erased simultaneously. To address these limitations, we introduce UnHype, a framework that incorporates hypernetworks into single- and multi-concept LoRA training. The proposed architecture can be directly plugged into Stable Diffusion as well as modern flow-based text-to-image models, where it demonstrates stable training behavior and effective concept control. During inference, the hypernetwork dynamically generates adaptive LoRA weights based on the CLIP embedding, enabling more context-aware, scalable unlearning. We evaluate UnHype across several challenging tasks, including object erasure, celebrity erasure, and explicit content removal, demonstrating its effectiveness and versatility. See the code on GitHub: https://github.com/gmum/UnHype.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。