arXiv:2506.23347cs.CV2025-06ICCV被引 3

用连续量化让自回归模型直接翻译图像,一步生成效果更好。

CycleVAR: Repurposing Autoregressive Model for Unsupervised One-Step Image Translation

  • 用Softmax重构码本选择,让梯度能顺畅流动
  • 并行一步生成比多步迭代更快更准
  • 适合需要快速高质量图像转换的场景

现有条件自回归图像生成方法表现良好,但在无监督图像翻译中潜力未被充分挖掘,因缺乏跨域对应关系。传统基于向量量化框架的离散量化会破坏变分自编码器解码器与因果Transformer之间的梯度流,阻碍图像空间对抗训练的端到端优化。为此,我们提出软化松弛量化(Softmax Relaxed Quantization),将码本选择重构成连续概率混合过程,保持梯度传播。在此可微基础上,我们提出CycleVAR,将图像到图像翻译重构为以源图像多尺度特征作为上下文提示的视觉自回归生成,类似语言模型的前缀条件。CycleVAR支持两种生成模式:(1) 串行多步生成,实现跨尺度迭代精炼;(2) 并行一步生成,在单次前向传播中同步输出所有分辨率结果。实验表明,在无监督场景下,并行一步生成在推理速度和翻译质量上均优于串行多步模式。定量与定性结果均显示,CycleVAR超越先前最先进模型,如CycleGAN-Turbo。

原文摘要 · Abstract (English)

The current conditional autoregressive image generation methods have shown promising results, yet their potential remains largely unexplored in the practical unsupervised image translation domain, which operates without explicit cross-domain correspondences. A critical limitation stems from the discrete quantization inherent in traditional Vector Quantization-based frameworks, which disrupts gradient flow between the Variational Autoencoder decoder and causal Transformer, impeding end-to-end optimization during adversarial training in image space. To tackle this issue, we propose using Softmax Relaxed Quantization, a novel approach that reformulates codebook selection as a continuous probability mixing process via Softmax, thereby preserving gradient propagation. Building upon this differentiable foundation, we introduce CycleVAR, which reformulates image-to-image translation as image-conditional visual autoregressive generation by injecting multi-scale source image tokens as contextual prompts, analogous to prefix-based conditioning in language models. CycleVAR exploits two modes to generate the target image tokens, including (1) serial multi-step generation, enabling iterative refinement across scales, and (2) parallel one-step generation synthesizing all resolution outputs in a single forward pass. Experimental findings indicate that the parallel one-step generation mode attains superior translation quality with quicker inference speed than the serial multi-step mode in unsupervised scenarios. Furthermore, both quantitative and qualitative results indicate that CycleVAR surpasses previous state-of-the-art unsupervised image translation models, \textit{e}.\textit{g}., CycleGAN-Turbo.

图像翻译自回归无监督生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。