用答案导向的思维链自动生成正负推理数据,提升多模态模型推理能力。
From Answers to Rationales: Self-Aligning Multimodal Reasoning with Answer-Oriented Chain-of-Thought
- 基于正确与错误答案生成正负推理路径,无需人工标注。
- 在多个模型上显著提升推理准确率,优于人工标注数据训练结果。
- 适合希望提升多模态模型推理鲁棒性的研究者和开发者。
实现多模态大语言模型(MLLMs)类人推理能力一直是研究目标。现有方法主要依赖人工标注或复杂系统合成正向推理过程,且常忽视负向推理,限制了模型在多模态推理中的泛化性与鲁棒性。为此,本文提出一种新框架:答案导向的思维链自对齐多模态推理(SMART)。SMART采用答案导向的思维链(AoT)提示,自动构建高质量数据。受人类证明策略启发,AoT利用正确与错误答案提取关联问题与答案的关键视觉信息。当输入正确答案时,模型生成强正向推理;当替换为错误答案时,模型产生看似合理但错误的推理路径,作为判别性负向推理。使用AoT生成数据训练的模型优于人工标注数据训练的结果,展现出更强的推理能力。SMART建立了一种迭代生成-优化机制,持续提升模型推理技能。实验表明,该框架可显著改进多种不同架构、参数量及预训练数据的MLLMs。代码已开源。
原文摘要 · Abstract (English)
Achieving human-like reasoning capabilities in Multimodal Large Language Models (MLLMs) has long been a goal. Current methods primarily focus on synthesizing positive rationales, typically relying on manual annotations or complex systems. Moreover, they often overlook negative reasoning, which limits the model's generalization ability and robustness in multimodal inference. To address this gap, we propose a novel framework: \textbf{S}elf-Aligning \textbf{M}ultimodal Reasoning with \textbf{A}nswer-O\textbf{r}iented Chain-of-\textbf{T}hought (SMART). SMART employs an answer-oriented chain-of-thought (AoT) prompt to automatically construct high-quality data. Drawing inspiration from human proof-based strategies, AoT leverages both correct and incorrect answers to extract key visual information that links questions and answers. When provided with correct answers, the model produces strong positive rationales. Conversely, when correct answers are replaced with incorrect alternatives, the model generates an erroneous yet compelling reasoning path, serving as a form of discriminative negative rationale. Models trained with AoT-generated data outperform those trained on manually annotated datasets, demonstrating superior reasoning capabilities. Consequently, SMART establishes an iterative generation-optimization method that continually enhances the model's reasoning skills. Experiments indicate that the SMART framework significantly improves various MLLMs, regardless of model architecture, parameter size, or pre-training dataset. The code is available at https://github.com/WentaoTan/SMART.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。