arXiv:2508.18673cs.CLcs.AI2025-08

用智能提示课程提升多模态推理,让模型逐步攻克难题。

Tailored Teaching with Balanced Difficulty: Elevating Reasoning in Multimodal Chain-of-Thought via Prompt Curriculum

  • 根据模型难易感知和题目本身复杂度,动态设计提示顺序
  • 在五个基准上显著提升多模态大模型推理性能
  • 适合想提升模型稳定性和泛化能力的研究者

多模态思维链(MCoT)提示的效果常受限于随机或手动选取的示例。这些示例未能考虑模型自身知识分布及任务内在复杂性,导致性能不佳且不稳定。为此,我们提出一种受‘因材施教、难度均衡’教育原则启发的新框架。将提示选择重构为提示课程设计问题:构建一组与模型当前能力相匹配的有序训练示例。方法融合两个互补信号:(1) 模型感知难度,通过主动学习中的预测分歧度量,反映模型自身的困惑;(2) 固有样本复杂度,独立于任何模型衡量每个问题-图像对的内在难度。联合分析两者,设计出兼顾难度分布与多样性的采样策略。在五个挑战性基准及多个主流多模态大模型(MLLMs)上的实验证明,该方法带来显著且一致的性能提升,大幅降低随机采样带来的性能波动,提供一种原理清晰、鲁棒性强的多模态推理增强方案。

原文摘要 · Abstract (English)

The effectiveness of Multimodal Chain-of-Thought (MCoT) prompting is often limited by the use of randomly or manually selected examples. These examples fail to account for both model-specific knowledge distributions and the intrinsic complexity of the tasks, resulting in suboptimal and unstable model performance. To address this, we propose a novel framework inspired by the pedagogical principle of "tailored teaching with balanced difficulty". We reframe prompt selection as a prompt curriculum design problem: constructing a well ordered set of training examples that align with the model's current capabilities. Our approach integrates two complementary signals: (1) model-perceived difficulty, quantified through prediction disagreement in an active learning setup, capturing what the model itself finds challenging; and (2) intrinsic sample complexity, which measures the inherent difficulty of each question-image pair independently of any model. By jointly analyzing these signals, we develop a difficulty-balanced sampling strategy that ensures the selected prompt examples are diverse across both dimensions. Extensive experiments conducted on five challenging benchmarks and multiple popular Multimodal Large Language Models (MLLMs) demonstrate that our method yields substantial and consistent improvements and greatly reduces performance discrepancies caused by random sampling, providing a principled and robust approach for enhancing multimodal reasoning.

多模态思维链提示工程推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。