arXiv:2507.07995cs.CVcs.AI2025-07NeurIPS被引 10

提出单次通过的自适应图像分块方法,按复杂度自动确定所需令牌数。

Single-pass Adaptive Image Tokenization for Minimum Program Search

  • 基于柯尔莫戈洛夫复杂度原理,单次前向传播预测最优令牌数
  • 在相同性能下比需多次搜索的方法更快,且能反映图像复杂度差异
  • 适合关注高效视觉表示与认知对齐的研究者

根据算法信息论,智能表示应将数据压缩为可重建内容的最短程序,具有低柯尔莫戈洛夫复杂度(KC)。现有视觉表征系统多采用固定长度表示,忽略输入间复杂度或熟悉度差异。近期自适应分块方法虽能分配可变长度表示,但通常需在测试时搜索多个编码以找到最优解。受柯尔莫戈洛夫复杂度启发,我们提出单次通过自适应分块器KARL,可在一次前向传播中预测图像所需的适当令牌数,并在近似达到其柯尔莫戈洛夫复杂度时停止。令牌数量作为最小描述长度的代理。KARL的训练过程类似倒置强化学习,学习根据期望重建质量条件性预测令牌停止点。该方法在性能上媲美最新自适应分块器,同时仅需单次通过。我们还给出了KARL的缩放定律,分析编码器/解码器规模、连续与离散分块等影响。此外,我们从概念上探讨了自适应图像分块与算法信息论的关系,研究了结构与噪声、分布内/外熟悉度等维度下的预测复杂度,结果与人类直觉一致。

原文摘要 · Abstract (English)

According to Algorithmic Information Theory (AIT) -- Intelligent representations compress data into the shortest possible program that can reconstruct its content, exhibiting low Kolmogorov Complexity (KC). In contrast, most visual representation learning systems use fixed-length representations for all inputs, ignoring variations in complexity or familiarity. Recent adaptive tokenization methods address this by allocating variable-length representations but typically require test-time search over multiple encodings to find the most predictive one. Inspired by Kolmogorov Complexity principles, we propose a single-pass adaptive tokenizer, KARL, which predicts the appropriate number of tokens for an image in a single forward pass, halting once its approximate KC is reached. The token count serves as a proxy for the minimum description length. KARL's training procedure closely resembles the Upside-Down Reinforcement Learning paradigm, as it learns to conditionally predict token halting based on a desired reconstruction quality. KARL matches the performance of recent adaptive tokenizers while operating in a single pass. We present scaling laws for KARL, analyzing the role of encoder/decoder size, continuous vs. discrete tokenization and more. Additionally, we offer a conceptual study drawing an analogy between Adaptive Image Tokenization and Algorithmic Information Theory, examining the predicted image complexity (KC) across axes such as structure vs. noise and in- vs. out-of-distribution familiarity -- revealing alignment with human intuition.

自适应分块信息论单次通过图像表示

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。