通过一次干预实现人脸生成公平性提升,无需重训练。
Learning Late, Guiding Early: Timestep-Decoupled Semantic Guidance for Fair Face Generation

- 在反向去噪初期一次性注入性别/种族引导信号
- 性别公平性差距降低98%,二元种族降低95%
- 无需微调模型,可直接接入现有扩散模型
合成人脸中的群体不平衡问题会传递到下游人脸识别系统,因此使用扩散模型生成数据时需考虑公平性。现有方法常需模型重训练、架构修改或反复引导。本文提出语义边界预测器(SBP),一种推理阶段的框架,在反向去噪过程中仅进行一次干预即可实现人口统计学引导。该方法基于观察:不同扩散步长的潜在表示具有不同语义角色——晚期潜在表示提供更强的人口统计分离能力,而早期潜在表示更利于语义干预。SBP从晚期潜在表示中学习线性语义边界,并仅在初始噪声潜在表示中应用一次,其余反向去噪过程保持不变。该方法无需对底层潜在扩散模型进行重训练或微调,也不依赖外部平衡数据集。在CelebA-HQ上的实验表明,该方法显著提升了人口统计公平性:性别公平性差距降低98%,二元种族降低95%,四类种族降低15%,同时保持各群体间感知图像质量一致。由于其单次推理策略和模型无关设计,SBP计算开销极小,可轻松集成至现有预训练潜在扩散模型。
原文摘要 · Abstract (English)
Demographic imbalance in synthetic face generation can propagate to downstream face recognition systems, making fairness an important consideration when diffusion models are used for data generation. Existing fairness-aware generation approaches often require model retraining, architectural modifications, or repeated guidance throughout the reverse diffusion process. In this work, we introduce Semantic Boundary Predictor (SBP), an inference-time framework that performs demographic guidance through a one-shot intervention during reverse denoising. Our approach is motivated by the observation that latent representations at different diffusion timesteps play distinct semantic roles: late-stage latents provide stronger demographic separability, whereas early-stage latents offer greater flexibility for semantic intervention. SBP leverages this timestep decoupling by learning linear semantic boundaries from late-stage latent representations while applying them only once at the initial noisy latent, allowing the remainder of the reverse denoising process to proceed unchanged. The method requires neither retraining nor fine-tuning of the underlying Latent Diffusion Model and operates without external balanced datasets. Experiments on CelebA-HQ demonstrate substantial improvements in demographic fairness, reducing fairness disparity by 98% for gender, 95% for binary race, and 15% for four-class race, while maintaining perceptual image quality across demographic groups. Owing to its one-shot inference strategy and model-agnostic design, SBP introduces only a small computational overhead and can be readily integrated with existing pre-trained latent diffusion models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。