让图像自适应生成不同长度的令牌，更贴近人类认知方式。
Adaptive Length Image Tokenization via Recurrent Allocation
- 通过递归迭代优化，动态调整图像令牌数量
- 令牌数量可在32到256之间变化，与图像复杂度匹配
- 适合追求高效视觉表示和可解释性的研究者
当前视觉系统对图像采用固定长度的表示，无论其信息量如何。这与人类智能及大型语言模型基于熵、上下文和熟悉度分配不同容量表征的方式形成对比。受此启发，我们提出一种学习二维图像可变长度令牌表示的方法。采用编码器-解码器架构，通过多轮递归处理，将2D图像令牌逐步提炼为1D潜在令牌。每轮迭代中，优化2D令牌，更新已有1D潜在令牌，并根据需要添加新令牌以自适应提升表征能力。该方法可将图像压缩为32至256个不等的令牌。通过重建损失和FID指标验证，令牌数量与图像熵、熟悉度及下游任务需求一致。递归处理过程表现出令牌特化现象，暗示具备对象或部件发现潜力。
原文摘要 · Abstract (English)
Current vision systems typically assign fixed-length representations to images, regardless of the information content. This contrasts with human intelligence - and even large language models - which allocate varying representational capacities based on entropy, context and familiarity. Inspired by this, we propose an approach to learn variable-length token representations for 2D images. Our encoder-decoder architecture recursively processes 2D image tokens, distilling them into 1D latent tokens over multiple iterations of recurrent rollouts. Each iteration refines the 2D tokens, updates the existing 1D latent tokens, and adaptively increases representational capacity by adding new tokens. This enables compression of images into a variable number of tokens, ranging from 32 to 256. We validate our tokenizer using reconstruction loss and FID metrics, demonstrating that token count aligns with image entropy, familiarity and downstream task requirements. Recurrent token processing with increasing representational capacity in each iteration shows signs of token specialization, revealing potential for object / part discovery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。