用简短推理和压缩视觉信息提升视频理解效率
Rethinking Chain-of-Thought Reasoning for Videos
- 用少量视觉标记+简洁推理链替代长链条人类式思考
- 在多个基准上表现不输长推理模型，速度大幅提升
- 无需人工标注或微调，适合实际部署场景
链式思维（CoT）在自然语言处理中表现优异，近期多模态大模型也将其拓展至视频推理。但现有方法依赖冗长的推理过程和大量视觉输入。基于我们基准测试的观察，我们提出：精简推理结合较少视觉标记即可实现高效视频推理。为此，我们设计并验证了一种高效的后训练与推理框架，使视频多模态大模型可在压缩视觉特征上生成简短推理路径后再作答。该方法显著提升推理效率，在多个基准上保持竞争力，且无需人工CoT标注或监督微调。结果表明，通用视频推理并不需要冗长的人类式思考，简洁推理既有效又高效。代码将开源于https://github.com/LaVi-Lab/Rethink_CoT_Video。
原文摘要 · Abstract (English)
Chain-of-thought (CoT) reasoning has been highly successful in solving complex tasks in natural language processing, and recent multimodal large language models (MLLMs) have extended this paradigm to video reasoning. However, these models typically build on lengthy reasoning chains and large numbers of input visual tokens. Motivated by empirical observations from our benchmark study, we hypothesize that concise reasoning combined with a reduced set of visual tokens can be sufficient for effective video reasoning. To evaluate this hypothesis, we design and validate an efficient post-training and inference framework that enhances a video MLLM's reasoning capability. Our framework enables models to operate on compressed visual tokens and generate brief reasoning traces prior to answering. The resulting models achieve substantially improved inference efficiency, deliver competitive performance across diverse benchmarks, and avoid reliance on manual CoT annotations or supervised fine-tuning. Collectively, our results suggest that long, human-like CoT reasoning may not be necessary for general video reasoning, and that concise reasoning can be both effective and efficient. Our code will be released at https://github.com/LaVi-Lab/Rethink_CoT_Video.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。