不重训练，通过优化文本嵌入实现扩散模型去偏。
Training-Free Debiasing of Diffusion Models via CLIP-Guided Denoising Optimization
- 在生成过程中直接优化条件文本嵌入，无需修改模型参数。
- 两阶段策略使图像性别与种族表示更公平，质量损失小。
- 适合需要快速部署公平生成能力的场景，如内容创作工具。
文本到图像的扩散模型虽具出色视觉质量，但中性提示仍常生成性别与种族刻板印象。现有方法受限于高昂的重训练成本或推理时干预带来的画质下降与语义偏差。本文提出无训练框架Text Embedding Steering（TES），通过在扩散过程中直接优化条件文本嵌入来缓解种族与性别偏见。实验表明，采用两阶段策略——早期全局对齐结合基于CLIP反馈的迭代去噪优化——可实现稳定可控的属性调节，且不改变模型参数。在Stable Diffusion上的大量实验证明，TES在公平性上优于现有无训练基线，同时保持优异的图像质量。结果表明，推理时文本嵌入优化是实现扩散模型公平生成的可行且可扩展方案。
原文摘要 · Abstract (English)
Text-to-image diffusion models achieve impressive visual quality, yet demographic bias remains a challenge, as neutral prompts consistently produce stereotypical representations across gender and race. Existing approaches remain limited by costly retraining or by inference-time interventions that often degrade image quality and semantic alignment. We propose Text Embedding Steering (TES), a training-free framework that mitigates demographic bias by directly optimizing conditional text embeddings during the diffusion process. We show that a two-stage strategy - early-stage global alignment followed by iterative denoising-time refinement with CLIP-based feedback - enables stable and controllable attribute steering without modifying model parameters. Extensive experiments on Stable Diffusion demonstrate that TES outperforms existing training-free baselines in fairness while maintaining competitive image quality. These results highlight that inference-time text embedding optimization is a practical and scalable solution for fairness-aware generation in diffusion models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。