根据图像复杂度自适应生成可变长度视觉令牌，提升生成质量与效率。
Soft Tail-dropping for Adaptive Visual Tokenization
- 基于图像复杂度动态决定输出令牌数量，结合保留概率控制序列长度。
- 在ImageNet-1k上，生成质量优于或媲美其他概率模型，且缩放性能稳定。
- 适合追求高效、可扩展的因果自回归视觉生成任务的研究者使用。
我们提出软尾部丢弃自适应分词器（STAT），一种一维离散视觉分词器，能根据图像结构复杂度和细节水平自适应选择每张图像的输出令牌数。STAT将图像编码为离散代码序列，并附带每个令牌的保留概率。除了标准自编码器目标外，还通过正则化使保留概率沿序列单调递减，并显式对齐其分布与图像级复杂度度量。结果是生成长度自适应的一维视觉令牌，天然兼容因果一维自回归（AR）视觉生成模型。在ImageNet-1k上，使用STAT增强的原始因果AR模型，在生成质量上达到或超过其他概率模型家族表现，同时展现出此前原始AR视觉生成中难以实现的有利缩放行为。
原文摘要 · Abstract (English)
We present Soft Tail-dropping Adaptive Tokenizer (STAT), a 1D discrete visual tokenizer that adaptively chooses the number of output tokens per image according to its structural complexity and level of detail. STAT encodes an image into a sequence of discrete codes together with per-token keep probabilities. Beyond standard autoencoder objectives, we regularize these keep probabilities to be monotonically decreasing along the sequence and explicitly align their distribution with an image-level complexity measure. As a result, STAT produces length-adaptive 1D visual tokens that are naturally compatible with causal 1D autoregressive (AR) visual generative models. On ImageNet-1k, equipping vanilla causal AR models with STAT yields competitive or superior visual generation quality compared to other probabilistic model families, while also exhibiting favorable scaling behavior that has been elusive in prior vanilla AR visual generation attempts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。