arXiv:2511.23334cs.CV2025-11中稿 · CVPR被引 4

提出新型视觉自回归模型,高效生成高质量图像。

Markovian Scale Prediction: A New Era of Visual Autoregressive Generation

  • 将多尺度预测建模为马尔可夫过程,用滑动窗口压缩历史信息。
  • 在ImageNet上FID降低10.5%,峰值内存减少83.8%。
  • 适合追求高效高质图像生成的研究者与应用开发者。

基于下一尺度预测的视觉自回归模型(VAR)重新激发了自回归图像生成的研究活力。尽管其全上下文依赖(即利用所有先前尺度进行下一尺度预测)通过完整信息流实现更稳定、全面的表征学习,但由此带来的计算效率低下和巨大开销严重制约了VAR的实际应用与可扩展性。为此,我们提出一种无需全上下文依赖的新VAR模型——Markov-VAR。该模型将多尺度预测重新构想为非全上下文马尔可夫过程,引入马尔可夫尺度预测机制:将每个尺度视为马尔可夫状态,并设计滑动窗口将部分先前尺度压缩为紧凑的历史向量,以弥补因非全上下文依赖导致的历史信息损失。将历史向量与马尔可夫状态结合,形成在马尔可夫过程中演化的动态状态。大量实验表明,Markov-VAR极为简洁却效果显著:相比ImageNet上的VAR模型,其在256×256下FID降低10.5%,在1024×1024下峰值内存消耗减少83.8%。我们认为,Markov-VAR可成为未来视觉自回归生成及其他下游任务的基础。

原文摘要 · Abstract (English)

Visual AutoRegressive modeling (VAR) based on next-scale prediction has revitalized autoregressive visual generation. Although its full-context dependency, i.e., modeling all previous scales for next-scale prediction, facilitates more stable and comprehensive representation learning by leveraging complete information flow, the resulting computational inefficiency and substantial overhead severely hinder VAR's practicality and scalability. This motivates us to develop a new VAR model with better performance and efficiency without full-context dependency. To address this, we reformulate VAR as a non-full-context Markov process, proposing Markov-VAR. It is achieved via Markovian Scale Prediction: we treat each scale as a Markov state and introduce a sliding window that compresses certain previous scales into a compact history vector to compensate for historical information loss owing to non-full-context dependency. Integrating the history vector with the Markov state yields a representative dynamic state that evolves under a Markov process. Extensive experiments demonstrate that Markov-VAR is extremely simple yet highly effective: Compared to VAR on ImageNet, Markov-VAR reduces FID by 10.5% (256 $\times$ 256) and decreases peak memory consumption by 83.8% (1024 $\times$ 1024). We believe that Markov-VAR can serve as a foundation for future research on visual autoregressive generation and other downstream tasks.

自回归生成图像生成马尔可夫模型高效生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。