评测多模态大模型纠错能力，发现推理增强的瓶颈。
MMRefine: Unveiling the Obstacles to Robust Refinement in Multimodal Large Language Models
- 构建六类场景的多模态纠错基准
- 揭示模型在错误检测与修正中的性能短板
- 适合研究多模态推理与模型优化的学者
本文提出MMRefine，一个用于评估多模态大语言模型（MLLMs）错误修正能力的多模态精炼基准。随着推理阶段推理能力提升成为重点，该基准不仅比较精炼前后的最终准确率，更在六种不同场景下评估模型检测并纠正错误的能力。通过将错误分为六类，分析了多种开源与闭源MLLM在精炼表现上的差异，揭示了阻碍有效推理增强的关键瓶颈和影响因素。相关代码与数据集已公开于https://github.com/naver-ai/MMRefine。
原文摘要 · Abstract (English)
This paper introduces MMRefine, a MultiModal Refinement benchmark designed to evaluate the error refinement capabilities of Multimodal Large Language Models (MLLMs). As the emphasis shifts toward enhancing reasoning during inference, MMRefine provides a framework that evaluates MLLMs' abilities to detect and correct errors across six distinct scenarios beyond just comparing final accuracy before and after refinement. Furthermore, the benchmark analyzes the refinement performance by categorizing errors into six error types. Experiments with various open and closed MLLMs reveal bottlenecks and factors impeding refinement performance, highlighting areas for improvement in effective reasoning enhancement. Our code and dataset are publicly available at https://github.com/naver-ai/MMRefine.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。