arXiv:2502.02339cs.CL2025-02AAAI被引 20

用自动提取的思维模板提升多模态模型推理能力,无需训练即可生效。

AStar: Boosting Multimodal Reasoning with Automated Structured Thinking

  • 通过提取高阶推理模式生成轻量级思维卡片库,辅助模型决策。
  • 在MathVerse上达53.9%准确率,超越GPT-4o的50.2%;MathVision达32.7%,优于30.4%。
  • 可跨任务迁移,适配数学、视觉理解等场景,且与现有方法兼容。

多模态大模型在多个领域表现优异,但在复杂视觉推理任务中仍显不足。现有方法依赖显式搜索或后训练技术,前者计算效率低,后者需大量数据和资源且训练不稳定。为此,我们提出无训练的自动结构化思维框架AStar:引入新型“思维卡片”,从历史样本中抽象出高层推理模式,构建轻量级知识库。测试时,AStar动态检索最优思维卡片,并无缝融合模型内部隐式推理能力。相比以往方法,AStar避免昂贵的显式搜索与复杂后训练,实现更高效的推理。实验表明,该框架在MathVerse上达到53.9%准确率(超越GPT-4o的50.2%),在MathVision上达32.7%(优于GPT-4o的30.4%)。进一步分析显示,数学推理生成的思维卡片具有显著迁移性,可有效提升其他推理任务乃至通用视觉感知性能。AStar为测试阶段推理提供即插即用方案,兼容其他后训练技术,是现有方法的重要补充。

原文摘要 · Abstract (English)

Multimodal large language models excel across diverse domains but struggle with complex visual reasoning tasks. To enhance their reasoning capabilities, current approaches typically rely on explicit search or post-training techniques. However, search-based methods suffer from computational inefficiency due to extensive solution space exploration, while post-training methods demand substantial data, computational resources, and often exhibit training instability. To address these challenges, we propose \textbf{AStar}, a training-free, \textbf{A}utomatic \textbf{S}tructured \textbf{t}hinking paradigm for multimod\textbf{a}l \textbf{r}easoning. Specifically, we introduce novel ``thought cards'', a lightweight library of high-level reasoning patterns abstracted from prior samples. For each test problem, AStar adaptively retrieves the optimal thought cards and seamlessly integrates these external explicit guidelines with the model's internal implicit reasoning capabilities. Compared to previous methods, AStar eliminates computationally expensive explicit search and avoids additional complex post-training processes, enabling a more efficient reasoning approach. Extensive experiments demonstrate that our framework achieves 53.9\% accuracy on MathVerse (surpassing GPT-4o's 50.2\%) and 32.7\% on MathVision (outperforming GPT-4o's 30.4\%). Further analysis reveals the remarkable transferability of our method: thought cards generated from mathematical reasoning can also be applied to other reasoning tasks, even benefiting general visual perception and understanding. AStar serves as a plug-and-play test-time inference method, compatible with other post-training techniques, providing an important complement to existing multimodal reasoning approaches.

多模态推理思维模板零样本测试时优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。