用自引导机制解决流模型生成多样性不足问题
Don't Settle at the Mode! Mitigating Diversity Collapse in Pretrained Flow Models via Feature Self-Guidance

- 通过特征自引导分散批量生成时的内部特征
- 引入流形正则化使特征回归数据主干,保持生成多样性
- 无需训练和额外奖励模型,适合快速部署
当前先进的流模型在文本或图像提示下可生成高质量图像,但在相同条件下的多样本生成中易出现多样性崩溃。现有方法要么依赖有限有效的潜在空间引导,要么需外部奖励模型进行样本筛选,带来显著推理开销。本文提出一种高效、无需训练的自引导机制,通过在批量生成时分散模型内部特征,并引入流形正则化将这些特征投影回数据流形,确保生成多样性同时保持与输入条件的一致性。该方法可作为即插即用模块集成到预训练流模型中,仅增加微量推理成本。实验表明,在多步与少步文本到图像、深度到图像及参考图像生成等多个任务上,该方法均显著提升多样性,同时保持生成保真度。
原文摘要 · Abstract (English)
State-of-the-art flow models generate stunning images from text or image prompts. However, they suffer from diversity collapse when generating multiple samples under the same conditioning. Existing methods address this issue via either latent guidance, which has limited effectiveness, or sample selection, which relies on external reward models that incur significant inference-time overhead. In this work, we introduce an efficient, training-free self-guidance mechanism to mitigate diversity collapse without requiring additional reward models. Specifically, we disperse the internal features of the flow model during batch generation with feature self-guidance. Further, to keep the features close to the manifold, we introduce a manifold regularization step that projects these dispersed features back onto the data manifold, ensuring diverse generation without sacrificing alignment with the input conditions. Our method integrates seamlessly as a plug-and-play module into pretrained flow models, adding only a marginal inference cost. Experiments demonstrate significant improvements in diversity while preserving fidelity across several conditional flow models, including multi-step and few-step text-to-image, depth-to-image, and reference image generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。