arXiv:2511.21415cs.CV2025-11中稿 · ECCV

提升视觉自回归模型生成多样性,不改模型也不加算力

DiverseVAR: Balancing Diversity and Quality of Next-Scale Visual Autoregressive Models

  • 测试时注入文本嵌入噪声,刺激生成更多样图像
  • 结合多尺度重构技术,使多样性提升同时质量下降可控
  • 适合追求生成多样性的图像生成应用开发者

我们提出 DiverseVAR,一种在不重新训练、微调或增加计算开销的前提下,提升文本条件视觉自回归模型(VAR)测试阶段多样性的框架。尽管 VAR 模型近期已成为图像生成中与扩散模型和流模型相竞争的强大方法,但其存在关键缺陷:即使面对简单提示,也常生成几乎相同的图像。这一问题长期被对图像质量的关注所掩盖。我们通过两阶段策略在测试时解决该问题:首先受扩散模型多样性增强启发,在文本嵌入中注入噪声,虽能提升多样性但会显著降低图像质量;为保持质量,我们提出“尺度旅行”(scale-travel)——一种受扩散模型时间旅行策略启发的新型潜在空间精炼技术。具体地,利用多尺度自编码器提取粗粒度令牌,实现中间阶段生成的恢复。大量实验表明,结合文本嵌入噪声注入与尺度旅行精炼可显著提升多样性,同时最小化图像质量损失,在多样性-质量权衡上达到新的帕累托前沿。

原文摘要 · Abstract (English)

We introduce DiverseVAR, a framework that enhances the diversity of text-conditioned visual autoregressive models (VAR) at test time without requiring retraining, fine-tuning, or substantial computational overhead. While VAR models have recently emerged as strong competitors to diffusion and flow models for image generation, they suffer from a critical limitation in diversity, often producing nearly identical images even for simple prompts. This issue has largely gone unnoticed amid the predominant focus on image quality. We address this limitation at test time in two stages. First, inspired by diversity enhancement techniques in diffusion models, we propose injecting noise into the text embedding. This introduces a trade-off between diversity and image quality: as diversity increases, the image quality sharply declines. To preserve quality, we propose scale-travel: a novel latent refinement technique inspired by time-travel strategies in diffusion models. Specifically, we use a multi-scale autoencoder to extract coarse-scale tokens that enable us to resume generation at intermediate stages. Extensive experiments show that combining text-embedding noise injection with our scale-travel refinement significantly enhances diversity while minimizing image-quality degradation, achieving a new Pareto frontier in the diversity-quality trade-off. Project page: https://diverse-var.github.io

视觉生成多样性提升自回归模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。