提升视觉自回归模型生成多样性，无需额外训练
Diversity Has Always Been There in Your Visual Autoregressive Models
- 通过调控特征图的关键成分，重构生成多样性
- 在不损失图像质量的前提下，显著提升输出变异性
- 适合追求高多样性生成的图像建模研究者
视觉自回归（VAR）模型因其新颖的逐尺度预测范式，相比传统多步自回归和扩散模型，在推理效率和图像质量方面表现优异。然而，尽管效率高，VAR模型常面临多样性坍缩问题，类似于少步蒸馏扩散模型中的现象。本文提出DiverseVAR，一种简单高效的方法，可在无需额外训练的情况下恢复VAR模型的生成多样性。分析表明，特征图中的关键成分是早期尺度多样性形成的核心因素。通过抑制模型输入中的关键成分、增强输出中的对应成分，DiverseVAR有效释放了VAR模型固有的生成潜力，同时保持高质量合成能力。实验结果表明，该方法显著提升了生成多样性，且对性能影响可忽略。代码将公开于https://github.com/wangtong627/DiverseVAR。
原文摘要 · Abstract (English)
Visual Autoregressive (VAR) models have recently garnered significant attention for their innovative next-scale prediction paradigm, offering notable advantages in both inference efficiency and image quality compared to traditional multi-step autoregressive (AR) and diffusion models. However, despite their efficiency, VAR models often suffer from the diversity collapse i.e., a reduction in output variability, analogous to that observed in few-step distilled diffusion models. In this paper, we introduce DiverseVAR, a simple yet effective approach that restores the generative diversity of VAR models without requiring any additional training. Our analysis reveals the pivotal component of the feature map as a key factor governing diversity formation at early scales. By suppressing the pivotal component in the model input and amplifying it in the model output, DiverseVAR effectively unlocks the inherent generative potential of VAR models while preserving high-fidelity synthesis. Empirical results demonstrate that our approach substantially enhances generative diversity with only neglectable performance influences. Our code will be publicly released at https://github.com/wangtong627/DiverseVAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。