让视频大模型摆脱语言偏见，靠动态推理路线提升视觉理解
VISTA: Mitigating Semantic Inertia in Video-LLMs via Training-Free Dynamic Chain-of-Thought Routing
- 不训练新模型，通过动态路由改变推理路径
- 在Egochema上提升9.3%，VideoEspresso上提升5.6%
- 适合需要高精度视频理解的开发者和研究者
近期大型语言模型已实现系统2式推理，但应用于视频理解仍面临挑战。现有研究多归因于感知能力不足，但我们实证发现，视频大模型存在一种认知错位——语义惯性，即模型会压制有效视觉证据，偏向主流语言先验。为此，我们提出VISTA，一种无需训练的框架，通过动态路由推理路径，并将隐含视觉特征显式化为文本锚点，有效抵消参数化知识影响。此外，引入潜在推理一致性机制以减少随机幻觉。VISTA在多个基准测试中表现优异，在Egochema上比基线模型提升9.3%，在VideoEspresso上提升5.6%，达到甚至超越更大、更封闭的模型水平。代码将很快开源。
原文摘要 · Abstract (English)
Recent advancements in Large Language Models have successfully transitioned towards System 2 reasoning, yet applying these paradigms to video understanding remains challenging. While prevailing research attributes failures in Video-LLMs to perceptual limitations, our empirical analysis reveals a cognitive misalignment termed Semantic Inertia, where models suppress valid visual evidence in favor of dominant language priors. To rectify this, we propose VISTA, a training-free framework designed to align perception with logical deduction. By dynamically routing inference paths and materializing implicit visual features into explicit textual anchors, our approach effectively counterbalances the influence of parametric knowledge. Furthermore, we incorporate a Latent Reasoning Consensus mechanism to mitigate stochastic hallucinations. VISTA showed outstanding results on a wide range of benchmarks, and outperforms its base model by 9.3% on Egochema and 5.6% on VideoEspresso, rivalling or even surpassing larger and proprietary models. Our codebase will be publicly available soon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。