通过分步校准信心,提升多模态模型的推理边界感知能力。
MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibration
- 在每一步推理中加入文本与跨模态自奖励信号,评估信心水平。
- 实测降低7.5%信心校准误差,任务性能最高提升8.3%。
- 适合关注多模态推理可信度与幻觉控制的研究者。
近年来,多模态大语言模型(MLLMs)虽取得显著进展,但在多层级(如感知、推理)和多粒度(如多步推理链)推理方面仍面临挑战。现有模型信心估计多聚焦整体输出,未能评估每一步推理的信心,导致幻觉逐级放大。本文提出MMBoundary框架,通过推理步骤信心校准,增强MLLM的知识边界感知能力。方法上,引入互补的文本与跨模态自奖励信号,对每一步推理进行信心估计;先通过监督微调初始化信心表达,再利用多奖励函数强化学习进一步对齐模型知识并校准每步信心,提升推理链自我修正能力。实验表明,该方法在多个领域数据集和指标上显著优于现有方法,平均降低7.5%的多模态信心校准误差,任务性能最高提升8.3%。
原文摘要 · Abstract (English)
In recent years, multimodal large language models (MLLMs) have made significant progress but continue to face inherent challenges in multimodal reasoning, which requires multi-level (e.g., perception, reasoning) and multi-granular (e.g., multi-step reasoning chain) advanced inferencing. Prior work on estimating model confidence tends to focus on the overall response for training and calibration, but fails to assess confidence in each reasoning step, leading to undesirable hallucination snowballing. In this work, we present MMBoundary, a novel framework that advances the knowledge boundary awareness of MLLMs through reasoning step confidence calibration. To achieve this, we propose to incorporate complementary textual and cross-modal self-rewarding signals to estimate confidence at each step of the MLLM reasoning process. In addition to supervised fine-tuning MLLM on this set of self-rewarded confidence estimation signal for initial confidence expression warm-up, we introduce a reinforcement learning stage with multiple reward functions for further aligning model knowledge and calibrating confidence at each reasoning step, enhancing reasoning chain self-correction. Empirical results show that MMBoundary significantly outperforms existing methods across diverse domain datasets and metrics, achieving an average of 7.5% reduction in multimodal confidence calibration errors and up to 8.3% improvement in task performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。