通过测试时动态调度提升视觉自回归生成质量,不增加训练成本。
TTS-VAR: A Test-Time Scaling Framework for Visual Auto-Regressive Generation
- 将生成过程视为路径搜索,动态调整批大小以平衡效率与探索。
- 在粗尺度用聚类保持结构多样性,细尺度用潜力评分重采样优选样本。
- 在Infinity模型上提升8.7%生成评估得分,适合高效生成场景研究者。
扩展视觉生成模型对现实内容创作至关重要,但训练和计算成本高昂。相比之下,测试时扩展因资源效率高且性能出色而受到关注。本文提出TTS-VAR,首个面向视觉自回归(VAR)模型的通用测试时扩展框架,将生成过程建模为路径搜索问题。为动态平衡计算效率与探索能力,我们引入了在因果生成过程中自适应递减的批量大小调度策略。此外,受VAR模型分层粗到细多尺度生成启发,框架集成两个关键组件:(i) 在粗尺度,生成标记难以评估,可能导致劣质样本被误接受或优质样本被误拒。鉴于粗尺度包含充分结构信息,我们提出基于聚类的多样性搜索,通过语义特征聚类保留结构多样性,实现后期对高潜力样本的选择;(ii) 在细尺度,基于重采样的潜力选择利用多尺度生成历史定义的奖励函数作为潜力评分,优先选取有前景的候选样本。在强大VAR模型Infinity上的实验显示,生成评估得分显著提升8.7%(从0.69升至0.75)。关键洞察表明,早期阶段的结构特征对最终质量影响显著,且重采样效果随生成尺度变化。代码已公开于https://github.com/ali-vilab/TTS-VAR。
原文摘要 · Abstract (English)
Scaling visual generation models is essential for real-world content creation, yet requires substantial training and computational expenses. Alternatively, test-time scaling has garnered growing attention due to resource efficiency and promising performance. In this work, we present TTS-VAR, the first general test-time scaling framework for visual auto-regressive (VAR) models, modeling the generation process as a path searching problem. To dynamically balance computational efficiency with exploration capacity, we first introduce an adaptive descending batch size schedule throughout the causal generation process. Besides, inspired by VAR's hierarchical coarse-to-fine multi-scale generation, our framework integrates two key components: (i) At coarse scales, we observe that generated tokens are hard for evaluation, possibly leading to erroneous acceptance of inferior samples or rejection of superior samples. Noticing that the coarse scales contain sufficient structural information, we propose clustering-based diversity search. It preserves structural variety through semantic feature clustering, enabling later selection on samples with higher potential. (ii) In fine scales, resampling-based potential selection prioritizes promising candidates using potential scores, which are defined as reward functions incorporating multi-scale generation history. Experiments on the powerful VAR model Infinity show a notable 8.7% GenEval score improvement (from 0.69 to 0.75). Key insights reveal that early-stage structural features effectively influence final quality, and resampling efficacy varies across generation scales. Code is available at https://github.com/ali-vilab/TTS-VAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。