arXiv:2606.28864cs.CV2026-06被引 1

测试时扩展让视觉语言模型在不改权重下提升性能,但效果受模型大小和计算量影响。

On Test-Time Scaling for Vision-Language Models

论文配图:On Test-Time Scaling for Vision-Language Models
图 1 · 摘自论文原文
  • 通过九种测试时扩展方法,评估多模型在六项基准上的表现。
  • 小模型经扩展后性能最高提升30%,甚至超过大模型。
  • 图像信息早期被编码,后续推理主要依赖文本,图像贡献下降。

测试时扩展是一种在不改变模型权重的前提下,通过增加推理时计算量来提升大模型性能的范式。尽管该方法在大型语言模型(LLMs)中已得到广泛研究,但在大型视觉语言模型(LVLMs)中的适用性仍缺乏充分探索与分析,尤其对方法是否有效、何时有效以及效果程度尚无系统研究。本文首次对LVLMs进行系统的测试时扩展研究,涵盖多个模型及规模、九种测试时扩展方法,以及六个多样化的基准测试。主要发现包括:1)与以往结论不同,小型且性能优异的模型在测试时扩展下获益最大,性能提升可达约30%,达到甚至超越大型模型水平;2)当分配的计算量超过必要值时,LVLMs会失去注意力焦点;3)视觉信息在推理链早期即被编码,此后推理过程主要由纯文本驱动,图像标记的贡献显著下降。此外,我们还进行了全局与细粒度的推理链质量与信息充分性分析。整体而言,研究结果为LVLMs在科研与工业部署中提供了实用指导与深入洞察。

原文摘要 · Abstract (English)

Test-time scaling is a paradigm where large models use additional compute at inference to achieve better performance, without changing model weights. While it has been widely studied for Large Language Models (LLMs), its applicability to Large Vision-Language Models (LVLMs) remains less explored and analyzed, with limited analysis of whether, when, and to what extent these approaches transfer to LVLMs. In this work, we ask a simple but fundamental question: can conventional test-time scaling methods developed for LLMs be directly applied to LVLMs? We present the first comprehensive study of test-time scaling for LVLMs, spanning multiple models and model sizes, nine test-time scaling methods, and six diverse benchmarks. Our main findings is that 1) different from previous findings, small, well-performing models benefit the most from test-time scaling, enabling performance improvements of up to around 30\%, reaching large models performance, and often outperforming them, 2) LVLMs lose focus when given more compute than necessary, and 3) Visual information is encoded early in the reasoning chain, after which the chain is dominated by text-only reasoning and the contribution of image tokens drops significantly. Finally, we also provide a global and fine-grained analysis on the quality and information sufficiency of the reasoning chains produced. Overall, our findings and analysis provide practical guidance and insights into LVLMs and their deployment in research and industry.

视觉语言模型测试时扩展推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。