将图像压缩为高语义1维令牌，提升重建与生成效果
Semantic One-Dimensional Tokenizer for Image Reconstruction and Generation
- 用2D到1D的映射方式生成带语义的离散令牌
- 重建精度超越现有方法，令牌规模更紧凑
- 适合需要高效图像生成与表示的任务
基于潜在空间的视觉生成模型已取得显著进展，凸显了视觉分词的重要性。将图像映射到潜在空间可提升效率，并支持多模态对齐以拓展下游任务。现有视觉分词器主要将图像映射为固定的二维空间网格，聚焦像素级恢复，难以捕捉具有紧凑全局语义的表征。为此，我们提出SemTok——一种语义一维分词器，将二维图像压缩为具高层次语义的一维离散令牌。SemTok在图像重建上达到新基准，以极简令牌实现优异保真度。其核心在于三项创新：2D到1D的分词方案、语义对齐约束与两阶段生成训练策略。基于SemTok构建的掩码自回归生成框架，在下游图像生成任务中表现显著提升。实验验证了该语义一维分词的有效性。代码将开源。
原文摘要 · Abstract (English)
Visual generative models based on latent space have achieved great success, underscoring the significance of visual tokenization. Mapping images to latents boosts efficiency and enables multimodal alignment for scaling up in downstream tasks. Existing visual tokenizers primarily map images into fixed 2D spatial grids and focus on pixel-level restoration, which hinders the capture of representations with compact global semantics. To address these issues, we propose \textbf{SemTok}, a semantic one-dimensional tokenizer that compresses 2D images into 1D discrete tokens with high-level semantics. SemTok sets a new state-of-the-art in image reconstruction, achieving superior fidelity with a remarkably compact token representation. This is achieved via a synergistic framework with three key innovations: a 2D-to-1D tokenization scheme, a semantic alignment constraint, and a two-stage generative training strategy. Building on SemTok, we construct a masked autoregressive generation framework, which yields notable improvements in downstream image generation tasks. Experiments confirm the effectiveness of our semantic 1D tokenization. Our code will be open-sourced.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。