提出原生视觉分词法，让图像生成更连贯。
NativeTok: Native Visual Tokenization for Improved Image Generation
- 在分词阶段强制引入因果依赖关系，避免生成混乱。
- 新框架在多个数据集上实现高效重建，且生成质量更高。
- 适合关注图像生成连贯性与训练效率的研究者。
基于向量量化（VQ）的图像生成通常采用两阶段流程：先用分词器将图像编码为离散标记，再由生成模型学习这些标记间的依赖关系以实现重建。然而，第一阶段的分词改进未必能提升第二阶段的生成效果，因现有方法未约束标记之间的依赖关系。这种不匹配导致生成模型需从无序分布中学习，引发偏差和低连贯性。为此，我们提出原生视觉分词（NativeTok），在分词过程中显式施加因果依赖。该框架包含：(1) 用于潜在图像建模的元图像变换器（MIT），(2) 混合因果专家变换器（MoCET），其中每个轻量级专家块基于先前标记和潜在特征生成单一标记。我们还设计了分层原生训练策略，仅更新新增专家块，保障训练效率。大量实验证明 NativeTok 的有效性。
原文摘要 · Abstract (English)
VQ-based image generation typically follows a two-stage pipeline: a tokenizer encodes images into discrete tokens, and a generative model learns their dependencies for reconstruction. However, improved tokenization in the first stage does not necessarily enhance the second-stage generation, as existing methods fail to constrain token dependencies. This mismatch forces the generative model to learn from unordered distributions, leading to bias and weak coherence. To address this, we propose native visual tokenization, which enforces causal dependencies during tokenization. Building on this idea, we introduce NativeTok, a framework that achieves efficient reconstruction while embedding relational constraints within token sequences. NativeTok consists of: (1) a Meta Image Transformer (MIT) for latent image modeling, and (2) a Mixture of Causal Expert Transformer (MoCET), where each lightweight expert block generates a single token conditioned on prior tokens and latent features. We further design a Hierarchical Native Training strategy that updates only new expert blocks, ensuring training efficiency. Extensive experiments demonstrate the effectiveness of NativeTok.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。