WinTok通过分离视觉理解与生成任务，实现高效统一的图像分词。
WinTok: A Win-Win Hybrid Tokenizer via Decomposing Visual Understanding and Generation with Transferable Tokens
- 将语义与像素两类令牌分离，避免任务冲突
- 仅用5000万数据训练，分类准确率超基线11.2%
- 适合需要兼顾理解与生成的多任务视觉模型
构建统一的视觉分词器对弥合视觉理解与生成之间的差距至关重要。然而现有方法在两项任务间存在固有矛盾：单一令牌空间需同时支持高层语义抽象与低层像素重建。我们提出WinTok，一种简洁的混合分词器，通过显式解耦两个目标，实现双赢性能。WinTok在像素令牌外补充可学习的语义令牌，有效缓解跨任务干扰，且无需双分词器带来的计算开销。为进一步提升理解能力，引入非对称令牌蒸馏机制：语义令牌由任意视觉基础模型的预训练语义嵌入引导，继承强区分能力的同时保持灵活性。在10个挑战性基准上，WinTok在重建、理解与生成任务中均实现一致提升。仅使用5000万开源数据训练，其分类准确率比强基线UniTok高出11.2%，重建指标rFID达0.41，且训练数据远少于基线。代码已开源：https://github.com/markywg/WinTok。
原文摘要 · Abstract (English)
Building a unified visual tokenizer is essential for bridging the gap between visual understanding and generation. Yet existing approaches struggle with the inherent conflict between these tasks, as a single token space is forced to support both high-level semantic abstraction and low-level pixel reconstruction. We propose WinTok, a concise hybrid tokenizer that achieves a win-win performance by explicitly decoupling the two objectives. WinTok supplements pixel tokens with a set of learnable semantic tokens, effectively mitigating cross-task interference without incurring the computational overhead of dual tokenizers. To further enhance understanding capability, we introduce an asymmetric token distillation mechanism: the semantic tokens are guided by pretrained semantic embeddings from any visual foundation model, enabling them to inherit strong discriminative power while maintaining flexibility. Across 10 challenging benchmarks, WinTok delivers consistent improvements in reconstruction, understanding, and generation. Trained on only 50M open-source data, WinTok surpasses the strong baseline UniTok by 11.2% in classification accuracy and achieves a competitive reconstruction rFID of 0.41, despite using substantially less training data. Code is released at https://github.com/markywg/WinTok.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。