无需修改模型即可通用移除图像生成中的有害概念
Uni-AdaVD: Universal Concept Erasure for Visual Generation via Orthogonal Value Decomposition
- 通过正交值分解在推理时干预注意力机制
- 支持多种生成模型，单/多概念移除效果好
- 适合需要安全可控生成的AI应用开发者
视觉生成模型会从非清洗预训练数据中吸收不良概念，因此概念擦除对安全部署至关重要。现有方法通常依赖特定架构，难以在移除目标概念的同时保留非目标内容与生成先验。我们提出Uni-AdaVD，一种面向视觉生成的通用推理时概念擦除框架。该方法将多模态注意力的值空间视为统一干预空间，引入编码器感知的目标表示构建，实现跨异构文本编码器的目标语义定位。进一步结合正交值分解与自适应擦除偏移，抑制目标语义方向而不更新原始模型权重。在U-Net、DiT及自回归图像生成器，以及文生视频模型上进行的大量实验表明，Uni-AdaVD在单概念和多概念擦除任务中均表现优异，同时有效保留非目标先验。结果表明，Uni-AdaVD为现代视觉生成模型提供了高效且可适配的安全机制。代码已开源：https://github.com/QifanZhou/Uni-AdaVD。
原文摘要 · Abstract (English)
Visual generative models inevitably absorb undesirable concepts from uncurated pretraining data, making concept erasure essential for safe deployment. Existing erasure methods, however, are often architecture-specific and struggle to remove target concepts while preserving non-target content and generative priors. We present Uni-AdaVD, a universal inference-time concept erasure framework for visual generation. Uni-AdaVD treats the value space of multimodal attention as a unified intervention space and introduces encoder-aware target representation construction to localize target semantics across heterogeneous text encoders. It further combines orthogonal value decomposition with an adaptive erasing shift to suppress target semantic directions without updating the original model weights. Extensive experiments on U-Net-, DiT-, and autoregressive image generators, as well as text-to-video models, demonstrate strong performance on single- and multi-concept erasure while preserving non-target priors. These results suggest that Uni-AdaVD provides an efficient and adaptable safety mechanism for modern visual generative models. Our code is available at https://github.com/QifanZhou/Uni-AdaVD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。