让图像自动生成长短可变的结构化代码序列，更懂场景本质。
Structure over Pixels: Learning Variable-Length Visual Programs
- 基于四阶段课程学习，动态预测每张图应生成多长的代码序列。
- 代码长度随场景复杂度自动增长，提升对结构信息的捕捉能力。
- 适合需要理解视觉结构的下游任务，如场景解析与生成。
离散视觉分词器将图像转换为有序代码序列，为场景结构描述提供自然表示。然而现有自适应分词器要么依赖后处理搜索，要么在预训练速率集合中选择，无法学习与模型和场景耦合的连续图像序列长度；且通常以像素重建为目标训练，侧重纹理而非结构。我们提出STROP，一种离散视觉分词器架构，能够形成结构化场景表示，并同步学习图像视觉程序的长度。通过使用局部率-失真探针监督冻结的DINOv3特征，采用四阶段课程学习，STROP优化一个专用长度头，可在一次前向传播中估计活跃前缀长度。通过绕过像素级重建梯度，码本完全由高层潜在表示质量塑造。程序长度随场景复杂度增长，下游密集预测迁移任务及直接观察学习到的代码词汇均显现出组合结构的迹象。
原文摘要 · Abstract (English)
Discrete visual tokenizers translate images into ordered sequences of codes, providing a natural representation for structural description of scenes. Yet existing adaptive tokenizers either require post-hoc search or select among a discrete set of pre-trained rates, rather than learning a continuous per-image sequence length coupled to the model and scene, and they typically train against pixel reconstruction, emphasizing texture rather than structure. We propose STROP, a discrete visual tokenizer architecture that forms structural scene representations and simultaneously learns how long an image's visual program should be. Using a four-phase curriculum supervised by local rate--distortion probes against frozen DINOv3 features, STROP optimizes a dedicated length head that estimates the active prefix length in a single forward pass. By bypassing pixel-level reconstruction gradients, the codebook is shaped entirely by the quality of higher-level latent representations. Program length grows with scene complexity, and signs of compositional structure emerge both in downstream dense-prediction transfer and in direct inspection of the learned code vocabulary.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。