解决流模型训练中低频主导问题,提升细节生成质量。
Balancing Frequencies and Pixels in Flow Matching

- 设计频域聚焦损失,均衡各频段学习信号
- 先频域后像素的双阶段训练,加速收敛40%
- 无需改架构,可直接替换现有损失函数
自然图像遵循1/f²谱分布:大部分能量集中在低频,而纹理、边缘等感知重要结构位于稀疏的高频区域。然而,像素空间重建目标对所有空间误差同等处理,导致低频主导优化信号,延缓细粒度细节的学习。本文识别出这一目标层面的频谱失衡是像素空间流模型训练的关键效率瓶颈。为此,提出焦点对数频率损失(f-loss),一种频谱平衡的目标函数,使各频段学习信号均等化,突出原本被像素目标忽略的高频成分。在此基础上,引入简单训练策略:早期强调频域学习以捕捉全频段信息,随后切换至标准像素空间v-loss进行空间精修。该平衡策略缓解了像素损失的低频偏差,使训练信号与模型演进需求一致。方法概念简洁,无需架构改动,可作为流匹配损失的即插即用替代方案。在多个模型规模下,收敛速度最高提升40%,并持续改善FID和感知保真度。代码与模型将公开。
原文摘要 · Abstract (English)
Natural images follow a $1/f^2$ spectral distribution: most signal energy lies in the low spatial frequencies, while the perceptually important structures such as textures and edges occupy sparse high-frequency bands. Pixel-space reconstruction objectives, however, treat all spatial errors uniformly, causing low frequencies to dominate the optimization signal and delaying the learning of fine-scale details. In this work, we identify this objective-level spectral imbalance as a key inefficiency in training pixel-space flow models. To address it, we propose a Focal Log-Frequency Loss (f-loss), a spectrally balanced objective that equalizes the learning signal across frequencies, emphasizing high-frequency components that are otherwise underrepresented in pixel-space objectives. Building on this, we introduce a simple training strategy that combines frequency and pixel supervision: we first emphasize frequency-domain learning early to capture all frequencies, and then transition to standard pixel-space v-loss for spatial refinement. This balancing mitigates the low-frequency bias of pixel losses and aligns the training signal with the evolving needs of the model. Our approach is conceptually simple, requires no architectural changes, and acts as a drop-in replacement for flow matching losses. Across multiple model scales, it accelerates convergence by up to 40% while consistently improving FID and perceptual fidelity. We will release code and models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。