不训练模型，用推理模型引导多模态理解，提升复杂推理能力。
ThinkOmni: Lifting Textual Reasoning to Omni-modal Scenarios via Guidance Decoding
- 用现成推理模型指导多模态大模型的生成过程。
- 在MathVista上达70.2分，在MMAU上达75.5分。
- 无需训练数据和调参，适合快速部署到各类多模态任务。
多模态推理对智能系统理解多样化数据源至关重要。现有多模态大语言模型（OLLM）虽能感知多种模态，但缺乏近期大型推理模型（LRM）的复杂推理能力。通过额外训练提升OLLM推理能力面临高质量数据、任务定制化及高计算成本等挑战。为此，我们提出ThinkOmni，一种无需训练、无需数据的框架，将文本推理能力迁移至多模态场景。ThinkOmni引入两大核心组件：1）LRM-as-a-Guide，利用现成的LRM引导OLLM解码；2）逐步对比缩放（Stepwise Contrastive Scaling），自适应平衡感知与推理信号，无需人工调参。在六个多模态推理基准上的实验表明，ThinkOmni持续提升性能，主结果在MathVista上达70.2，在MMAU上达75.5。整体而言，ThinkOmni为多模态推理提供了灵活通用的解决方案，并为推理能力的泛化与应用提供新视角。代码公开于https://github.com/1ranGuan/thinkomni。
原文摘要 · Abstract (English)
Omni-modal reasoning is essential for intelligent systems to understand and draw inferences from diverse data sources. While existing omni-modal large language models (OLLM) excel at perceiving diverse modalities, they lack the complex reasoning abilities of recent large reasoning models (LRM). However, enhancing the reasoning ability of OLLMs through additional training presents significant challenges, including the need for high-quality data, task-specific adaptation, and substantial computational costs. To address these limitations, we propose ThinkOmni, a training-free and data-free framework that lifts textual reasoning to omni-modal scenarios. ThinkOmni introduces two key components: 1) LRM-as-a-Guide, which leverages off-the-shelf LRMs to guide the OLLM decoding process; 2) Stepwise Contrastive Scaling, which adaptively balances perception and reasoning signals without manual hyperparameter tuning. Experiments on six multi-modal reasoning benchmarks demonstrate that ThinkOmni consistently delivers performance improvements, with main results achieving 70.2 on MathVista and 75.5 on MMAU. Overall, ThinkOmni offers a flexible and generalizable solution for omni-modal reasoning and provides new insights into the generalization and application of reasoning capabilities. Code is publicly available at https://github.com/1ranGuan/thinkomni
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。