arXiv:2601.07293cs.CV2026-01中稿 · PRCV 2025被引 1

通过采样空间重构,实现视觉自回归模型的推理时扩展,提升生成质量。

Inference-Time Scaling for Visual AutoRegressive modeling by Searching Representative Samples

  • 用核密度估计将离散潜在空间映射为准连续特征空间,突破离散约束。
  • 提出混合采样策略:顶k保质量,随机k保多样性,显著提升生成效果。
  • 适用于文本到图像和类别条件生成,适合关注生成质量的视觉建模研究者。

尽管推理时扩展在大型语言模型和扩散模型中显著提升了生成质量,但在向量量化(VQ)视觉自回归模型(VAR)中的应用仍属空白。本文提出VAR-Scaling,首个针对VAR的推理时扩展通用框架,解决了离散潜在空间无法进行连续路径搜索的关键挑战。我们发现VAR扩展呈现两类模式:通用模式与特定模式,其中后期特定模式会条件性优化早期通用模式。为克服VQ模型离散空间的障碍,我们通过核密度估计(KDE)将采样空间映射至准连续特征空间,高密度样本逼近稳定且高质量解。该变换使采样分布可有效导航。我们提出密度自适应混合采样策略:Top-k采样聚焦高密度区域以保持分布峰值处的质量,Random-k采样探索低密度区域以维持多样性并防止过早收敛。因此,VAR-Scaling在关键尺度上优化样本保真度,提升输出质量。在类别条件与文本到图像评估中,实验均显示推理过程有显著改进。代码已开源:https://github.com/WD7ang/VAR-Scaling。

原文摘要 · Abstract (English)

While inference-time scaling has significantly enhanced generative quality in large language and diffusion models, its application to vector-quantized (VQ) visual autoregressive modeling (VAR) remains unexplored. We introduce VAR-Scaling, the first general framework for inference-time scaling in VAR, addressing the critical challenge of discrete latent spaces that prohibit continuous path search. We find that VAR scales exhibit two distinct pattern types: general patterns and specific patterns, where later-stage specific patterns conditionally optimize early-stage general patterns. To overcome the discrete latent space barrier in VQ models, we map sampling spaces to quasi-continuous feature spaces via kernel density estimation (KDE), where high-density samples approximate stable, high-quality solutions. This transformation enables effective navigation of sampling distributions. We propose a density-adaptive hybrid sampling strategy: Top-k sampling focuses on high-density regions to preserve quality near distribution modes, while Random-k sampling explores low-density areas to maintain diversity and prevent premature convergence. Consequently, VAR-Scaling optimizes sample fidelity at critical scales to enhance output quality. Experiments in class-conditional and text-to-image evaluations demonstrate significant improvements in inference process. The code is available at https://github.com/WD7ang/VAR-Scaling.

视觉生成自回归模型推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。