用三阶段框架生成高质量多模态推理链,提升大模型推理能力
Training Multimodal Large Reasoning Models Needs Better Thoughts: A Three-Stage Framework for Long Chain-of-Thought Synthesis and Selection
- 多模型生成候选推理链,分两层筛选优质内容
- 在多个基准上显著超越基线,强化学习后效果更优
- 适合研究多模态推理与长链条思维生成的学者
大型推理模型(LRMs)通过长链式思维(CoT)在复杂推理任务中表现优异。但将其拓展至多模态推理仍面临挑战,主要源于异构模态融合复杂度高,以及高质量长CoT训练数据稀缺。现有多模态数据集和CoT生成方法普遍存在推理深度不足、模态转换错误及生成流程僵化等问题,限制了模型性能与稳定性。为此,本文提出SynSelect——一种面向多模态推理任务的三阶段合成-选择框架,用于生成高质量长CoT数据。该框架首先利用多个异构多模态大推理模型生成多样候选推理链,随后在实例与批量两个层级进行筛选,保留能有效提升模型推理能力的优质推理链。在多个多模态基准上的大量实验表明,基于SynSelect生成数据微调的模型显著优于基线,并在强化学习后训后进一步提升性能。结果验证了SynSelect作为提升多模态大推理模型推理能力的有效方法。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) have demonstrated remarkable performance on complex reasoning tasks through long Chain-of-Thought (CoT) reasoning. Extending these successes to multimodal reasoning remains challenging due to the increased complexity of integrating diverse input modalities and the scarcity of high-quality long CoT training data. Existing multimodal datasets and CoT synthesis methods still suffer from limited reasoning depth, modality conversion errors, and rigid generation pipelines, hindering model performance and stability. To this end, in this paper, we propose SynSelect, a novel three-stage Synthesis-Selection framework for generating high-quality long CoT data tailored to multimodal reasoning tasks. Specifically, SynSelect first leverages multiple heterogeneous multimodal LRMs to produce diverse candidate CoTs, and then applies both instance and batch level selection to filter high-quality CoTs that can effectively enhance the model's reasoning capabilities. Extensive experiments on multiple multimodal benchmarks demonstrate that models supervised fine-tuned on SynSelect-generated data significantly outperform baselines and achieve further improvements after reinforcement learning post-training. Our results validate SynSelect as an effective approach for advancing multimodal LRMs reasoning capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。