arXiv:2512.21276cs.CV2025-12

分步生成长序列图像,先低分辨率建模再局部超分,更快更清晰。

GriDiT: Factorized Grid-Based Diffusion for Efficient Long Image Sequence Generation

  • 先生成低分辨率序列,再逐帧超分,避免整体处理大张量
  • 在多个数据集上质量更高,推理速度至少快两倍
  • 无需额外监督,可跨领域泛化,适合长序列生成任务

现有深度学习方法通常将图像序列视为连续堆叠的帧张量,但这一表示是否仍是当前最优?本文针对生成模型提出新思路:将序列生成分解为两步——先在低分辨率下生成粗略序列,再对每帧独立进行高分辨率超分。模型仅在子采样帧构成的网格图像上训练,利用扩散变换器(DiT)的自注意力机制捕捉帧间相关性。该方法将2D图像生成器扩展为无需架构修改的低分辨率3D序列生成器,随后逐帧添加细节。相比现有方法,本方案在合成质量、序列连贯性、任意长度生成和推理效率方面均有提升,且训练数据使用更少。实验表明,该方法在多个数据集上均超越当前最优水平,推理速度至少快两倍,且能有效跨域泛化,无需额外先验或监督。

原文摘要 · Abstract (English)

Modern deep learning methods typically treat image sequences as large tensors of sequentially stacked frames. However, is this straightforward representation ideal given the current state-of-the-art (SoTA)? In this work, we address this question in the context of generative models and aim to devise a more effective way of modeling image sequence data. Observing the inefficiencies and bottlenecks of current SoTA image sequence generation methods, we showcase that rather than working with large tensors, we can improve the generation process by factorizing it into first generating the coarse sequence at low resolution and then refining the individual frames at high resolution. We train a generative model solely on grid images comprising subsampled frames. Yet, we learn to generate image sequences, using the strong self-attention mechanism of the Diffusion Transformer (DiT) to capture correlations between frames. In effect, our formulation extends a 2D image generator to operate as a low-resolution 3D image-sequence generator without introducing any architectural modifications. Subsequently, we super-resolve each frame individually to add the sequence-independent high-resolution details. This approach offers several advantages and can overcome key limitations of the SoTA in this domain. Compared to existing image sequence generation models, our method achieves superior synthesis quality and improved coherence across sequences. It also delivers high-fidelity generation of arbitrary-length sequences and increased efficiency in inference time and training data usage. Furthermore, our straightforward formulation enables our method to generalize effectively across diverse data domains, which typically require additional priors and supervision to model in a generative context. Our method consistently outperforms SoTA in quality and inference speed (at least twice-as-fast) across datasets.

图像生成扩散模型序列生成高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。