提出TokenBridge,让视觉生成同时拥有连续与离散标记的优势。
Bridging Continuous and Discrete Tokens for Autoregressive Visual Generation
- 用后训练量化将连续标记转为离散,解耦了分词与训练过程
- 在ImageNet上重建误差仅比纯连续方法高0.3%,生成质量相当
- 适合追求高质量生成且希望保持简单预测结构的研究者
自回归视觉生成模型通常依赖分词器将图像压缩为可逐个预测的标记。标记表示面临根本矛盾:离散标记虽便于使用标准交叉熵损失建模,但易丢失信息且分词器训练不稳定;连续标记更保真,却需复杂分布建模,使生成流程复杂化。本文提出TokenBridge,通过后训练量化将连续表征直接转为离散标记,解耦了分词器训练与离散化过程,实现连续标记的强大表征能力与离散标记的建模简便性。具体采用逐维度量化策略,独立离散化每个特征维度,并结合轻量级自回归预测机制高效建模庞大的离散标记空间。大量实验表明,该方法在重建与生成质量上与连续方法相当,同时使用标准分类预测。本工作证明,融合离散与连续范式可有效结合两者优势,为简化建模下的高质量视觉生成提供新方向。
原文摘要 · Abstract (English)
Autoregressive visual generation models typically rely on tokenizers to compress images into tokens that can be predicted sequentially. A fundamental dilemma exists in token representation: discrete tokens enable straightforward modeling with standard cross-entropy loss, but suffer from information loss and tokenizer training instability; continuous tokens better preserve visual details, but require complex distribution modeling, complicating the generation pipeline. In this paper, we propose TokenBridge, which bridges this gap by maintaining the strong representation capacity of continuous tokens while preserving the modeling simplicity of discrete tokens. To achieve this, we decouple discretization from the tokenizer training process through post-training quantization that directly obtains discrete tokens from continuous representations. Specifically, we introduce a dimension-wise quantization strategy that independently discretizes each feature dimension, paired with a lightweight autoregressive prediction mechanism that efficiently model the resulting large token space. Extensive experiments show that our approach achieves reconstruction and generation quality on par with continuous methods while using standard categorical prediction. This work demonstrates that bridging discrete and continuous paradigms can effectively harness the strengths of both approaches, providing a promising direction for high-quality visual generation with simple autoregressive modeling. Project page: https://yuqingwang1029.github.io/TokenBridge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。