让图像生成避开不想要的内容,自动调整去噪路径避免干扰。
Set You Straight: Auto-Steering Denoising Trajectories to Sidestep Unwanted Concepts
- 反向引导方向自动修正去噪轨迹,无需人工选锚点。
- 单/多概念擦除效果领先,生成图像质量高且无瑕疵。
- 适合需要内容安全的AI绘图应用,如广告与教育领域。
确保文生图模型的伦理部署需有效阻止有害或不当内容生成。现有微调方法存在局限:无锚点方法可能破坏采样轨迹,引发视觉伪影;有锚点方法依赖启发式选择锚概念。为此,我们提出ANT框架,通过在中后期去噪阶段反转无分类器引导的方向,实现对去噪轨迹的自动调控,从而精准规避不良概念,同时保持早期结构完整性。该方法构建了轨迹感知目标函数,无需依赖启发式锚点,可保留早期得分函数场的自然图像流形导向性。针对单概念擦除,提出增强型权重显著性图,精确识别影响最显著的关键参数,提升擦除效率与彻底性;针对多概念擦除,目标函数提供即插即用的通用解决方案,性能显著提升。大量实验表明,ANT在单/多概念擦除任务中均达到当前最优,生成高质量、安全的输出,且不损失生成保真度。代码已开源。
原文摘要 · Abstract (English)
Ensuring the ethical deployment of text-to-image models requires effective techniques to prevent the generation of harmful or inappropriate content. While concept erasure methods offer a promising solution, existing finetuning-based approaches suffer from notable limitations. Anchor-free methods risk disrupting sampling trajectories, leading to visual artifacts, while anchor-based methods rely on the heuristic selection of anchor concepts. To overcome these shortcomings, we introduce a finetuning framework, dubbed ANT, which Automatically guides deNoising Trajectories to avoid unwanted concepts. ANT is built on a key insight: reversing the condition direction of classifier-free guidance during mid-to-late denoising stages enables precise content modification without sacrificing early-stage structural integrity. This inspires a trajectory-aware objective that preserves the integrity of the early-stage score function field, which steers samples toward the natural image manifold, without relying on heuristic anchor concept selection. For single-concept erasure, we propose an augmentation-enhanced weight saliency map to precisely identify the critical parameters that most significantly contribute to the unwanted concept, enabling more thorough and efficient erasure. For multi-concept erasure, our objective function offers a versatile plug-and-play solution that significantly boosts performance. Extensive experiments demonstrate that ANT achieves state-of-the-art results in both single and multi-concept erasure, delivering high-quality, safe outputs without compromising the generative fidelity. Code is available at https://github.com/lileyang1210/ANT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。