将自回归建模引入流模型,提升长距离依赖捕捉能力。
ARFlow: Autoregressive Flow with Hybrid Linear Attention
- 用噪声等级构建因果序列,实现渐进式图像生成
- 256×256下达6.63 FID,使用指导后降至1.96 FID
- 适合追求高质量图像生成与高效注意力设计的研究者
流模型在逐步生成逼真图像方面表现优异,但通常难以捕捉生成过程中的长距离依赖关系,因其将前期所有信息压缩为一张退化的图像。为此,我们提出将自回归建模——擅长建模复杂高维联合分布——融入流模型。训练时,通过从同一语义类别中采样多张图像并施加不同层级的噪声,使噪声更高的图像作为噪声更低图像的因果前驱,从而让模型学习更广泛的类别级变化,同时保持流过程中的正确因果性。生成时,模型自回归地基于先前去噪步骤生成的图像进行条件建模,形成上下文连贯的生成轨迹。此外,我们设计了针对该方法的混合线性注意力机制,以提升计算效率。所提方法ARFlow在无分类器引导下于ImageNet 256×256上达到6.63 FID,使用分类器引导(引导强度1.5)后降至1.96 FID,优于此前流模型SiT的2.06 FID。大量消融实验验证了该建模策略与分块注意力设计的有效性。
原文摘要 · Abstract (English)
Flow models are effective at progressively generating realistic images, but they generally struggle to capture long-range dependencies during the generation process as they compress all the information from previous time steps into a single corrupted image. To address this limitation, we propose integrating autoregressive modeling -- known for its excellence in modeling complex, high-dimensional joint probability distributions -- into flow models. During training, at each step, we construct causally-ordered sequences by sampling multiple images from the same semantic category and applying different levels of noise, where images with higher noise levels serve as causal predecessors to those with lower noise levels. This design enables the model to learn broader category-level variations while maintaining proper causal relationships in the flow process. During generation, the model autoregressively conditions the previously generated images from earlier denoising steps, forming a contextual and coherent generation trajectory. Additionally, we design a customized hybrid linear attention mechanism tailored to our modeling approach to enhance computational efficiency. Our approach, termed ARFlow, achieves 6.63 FID scores on ImageNet at 256 * 256 without classifier-free guidance, reaching 1.96 FID with classifier-free guidance 1.5, outperforming the previous flow-based model SiT's 2.06 FID. Extensive ablation studies demonstrate the effectiveness of our modeling strategy and chunk-wise attention design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。