arXiv:2606.07185cs.CV2026-06

让图像分块自动决定所需数量,复杂图像多分块,简单图像少分块。

AdaTok: Self-Budgeting Image Tokenization with Quality-Preserving Dynamic Tokens

论文配图:AdaTok: Self-Budgeting Image Tokenization with Quality-Preserving Dynamic Tokens
图 1 · 摘自论文原文
  • 根据图像复杂度动态分配token数,自适应调整输入长度。
  • 在ImageNet上用118个平均token达到rFID 1.50,优于固定256个的基线。
  • 适合需要高效推理的视觉生成任务,如图像生成与压缩场景。

传统图像分块器对所有图像使用固定数量的分块,但视觉复杂度差异大,导致简单图像浪费资源、复杂图像信息不足。现有弹性分块器虽可变长输出,但分块数仍需外部设定或搜索。本文提出AdaTok,一种自预算的离散1D分块器,通过表示-分配协同设计实现单次遍历自动决定分块数。其核心是优先级表示学习(嵌套尾部掩码+多头LoRA解码头)和自适应分块分配(基于轻量级确定性组GRPO策略)。动态帕累托加权在训练中平衡保真度与效率,无需手动调参。在ImageNet-1K上,AdaTok-Full以256分块达rFID 1.31;AdaTok-Adaptive平均仅用约118分块即达rFID 1.50,优于同类方法。在自回归图像生成中,更短的自适应表示使吞吐量提升约2.1倍,证明分块数可作为内容相关的输出,而非固定超参数。

原文摘要 · Abstract (English)

Image tokenizers, from 2D grids to recent 1D sequences, typically encode every image with the same fixed number of tokens. Yet visual complexity is highly heterogeneous, so a uniform budget overspends on simple inputs and underserves complex ones. Existing elastic tokenizers expose variable-length reconstructions, but often leave token length as a deployment-time operating point, a search target, or an external prediction rather than an output of the tokenizer itself. In this work, we ask whether a discrete visual tokenizer can budget itself in one pass. Our central finding is that actionable elasticity requires a representation--allocation co-design: prefixes must remain decodable across budgets, and the tokenizer must learn which prefix each image needs. We propose AdaTok, a self-budgeting discrete 1D tokenizer. AdaTok combines Prioritized Representation Learning, which orders tokens with nested tail masking and resolves budget-dependent semantic shift through Multi-Head LoRA decoder heads, with Adaptive Token Allocation, which trains a lightweight deterministic-group GRPO policy over candidate budgets. Dynamic Pareto Weighting balances fidelity and efficiency during policy training without manual trade-off sweeps. On ImageNet-1K, AdaTok-Full reaches rFID 1.31 at 256 tokens, while AdaTok-Adaptive attains rFID 1.50 using only ~118 tokens on average, outperforming discrete 1D baselines at comparable budgets. In autoregressive image generation, the shorter adaptive representation yields ~2.1x throughput over a fixed 256-token decode, suggesting that visual token count can be learned as a content-conditioned output rather than set as a fixed hyperparameter.

图像分块自适应生成模型效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。