按频率分层生成图像,先建粗结构再补细节,提升高分辨率图像质量。
WaiT for the Signal: Simple Frequency-Aware Flow-Matching

- 用小波分解图像,分粗细频段分别生成,细频段等粗结构出现后再加入
- 在ImageNet 512x512上实现1.43的像素空间FID,采样计算量减少50%
- 适合追求高分辨率、纹理真实感的图像与视频生成研究者
随着图像生成模型向更高分辨率发展,全局一致性、局部细节和纹理保真度成为关键质量指标。然而,标准流匹配对所有空间频率一视同仁,忽略了高频成分远早于粗结构就趋于噪声的自然规律。本文提出WaiT,一种小波感知的图像Transformer,通过无损小波分解将生成过程分为粗频段与细频段。名如其实,高频部分等待信号:在粗结构生成前保持纯噪声,待结构浮现后才参与联合优化。由于标准FID因激进下采样丢失细粒度信息,我们引入更严格的三轴评估协议以衡量原生分辨率下的质量。在ImageNet 512x512上,WaiT达到1.43的像素空间FID,且在三轴上均处于帕累托最优,采样计算最多降低50%。使用最大2B模型,在512分辨率上取得1.3的新纪录,超越最强潜空间模型的纹理保真度,并可无缝扩展至高分辨率OpenImages及视频生成,在Kinetics-600上实现0.84的最优FVD,无需算法修改。
原文摘要 · Abstract (English)
As image generation models scale to ever higher resolutions, global coherence, local detail, and texture fidelity become critical axes for generation quality. However, standard flow matching treats all spatial frequencies uniformly, ignoring the natural frequency hierarchy where high-frequency bands become indistinguishable from pure noise far earlier than coarse structures. We introduce WaiT, a Wavelet-aware image Transformer that decomposes generation into coarse and fine bands via lossless wavelets. True to its name, the high-frequency bands wait for the signal: staying pure noise until coarse structure has emerged, then joining the flow for joint refinement. Since standard FID discards fine-grained detail through aggressive downsampling, we introduce a more stringent three-axis evaluation protocol to assess quality at native resolution. On ImageNet 512x512, WaiT achieves a pixel-space FID of 1.43 and is Pareto-optimal across all three axes, reducing sampling compute by up to 50%. With our largest 2B model, we set a new state-of-the-art FID of 1.3 for pixel-space models on ImageNet 512 resolution. Our formulation outperforms even the strongest latent-space models on texture fidelity, and scales seamlessly to high-resolution OpenImages and to video generation, achieving a state-of-the-art FVD of 0.84 on Kinetics-600 with no algorithmic modifications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。