用可变长度一维序列压缩图像，提升视觉语言模型效率与精度
RADIO1D: Elastic Representations for Condensed Vision Modeling
- 通过多教师知识蒸馏和自编码器设计，将图像转为可变长度1D序列
- 仅用一个令牌即可准确理解场景，支持更高效的图像检索
- 适配多模态任务，兼具高精度与低计算开销，适合部署优化
本文挑战了视觉语言模型（VLMs）必须依赖固定2D图像块特征的假设。分析微调后的视觉编码器发现，随着训练进行，表示逐渐变得抽象且空间一致性减弱。特别地，采用图像-文本对齐训练（如SigLIP2）的模型会生成少量专用令牌，有效总结全局图像内容。基于此，我们提出RADIO1D，利用多教师知识蒸馏与自编码器架构，将图像压缩为紧凑、可变长度的一维令牌序列。所得表示具备强层级摘要能力，在仅用一个令牌时即能实现精准场景理解，并支持更优的组合感知图像检索。在VLM中，RADIO1D通过调整令牌数量提供灵活的精度-效率权衡，在多个多模态基准上表现优异，同时降低计算开销并提升准确性。
原文摘要 · Abstract (English)
This paper challenges the assumption that vision-language models (VLMs) require fixed patch-based 2D vision features. Analyzing fine-tuned vision encoders, we find that representations become increasingly abstract and less spatially coherent during VLM training. Notably, models trained with image-text alignment (such as SigLIP2) develop a small number of specialized tokens that effectively summarize global image content. Building on this, we introduce RADIO1D, which compresses images into a compact, variable-length 1D token sequence using multi-teacher knowledge distillation and an autoencoder design. The resulting representations exhibit strong hierarchical summarization, enabling accurate scene understanding - even with a single token - and support improved composition-aware image retrieval. In VLMs, RADIO1D provides flexible accuracy-efficiency tradeoffs through adjustable token counts, delivering competitive performance on diverse multimodal benchmarks with lower computational overhead and better accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。