新基准测试揭示大模型视觉推理短板
Can MLLMs Reason Beyond Language? VisReason: A Comprehensive Benchmark for Vision-Centric Reasoning
- 构建10类1505个日常场景视觉推理题
- 人类与模型差距显著，现有策略效果有限
- 专测视觉驱动推理，非语言依赖任务
近期多模态大模型在视觉推理任务上表现强劲，但其性能究竟在多大程度上基于真实视觉证据仍不明确。本文提出VisReason，一个面向日常场景的视觉中心推理基准，包含10个类别共1505个问题，覆盖感知、结构与概念推理。评估显示，该基准带来了质的不同挑战，暴露出人类与当前多模态大模型间的显著差距，且测试时推理策略带来的收益有限。VisReason为评估超越语言的视觉中心推理提供了精准诊断工具。
原文摘要 · Abstract (English)
Recent multimodal large language models (MLLMs) achieve strong performance on visual reasoning benchmarks, yet it remains unclear to what extent such performance reflects reasoning directly grounded in visual evidence. We introduce VisReason, a benchmark for vision-centric reasoning in everyday scenarios where perception and inference are tightly coupled. VisReason contains 1,505 questions across 10 categories spanning perceptual, structural, and conceptual reasoning. Our evaluation shows that VisReason poses a qualitatively different challenge from existing benchmarks, exposing substantial gaps between humans and current MLLMs and revealing limited benefits from test-time reasoning strategies. VisReason offers a focused diagnostic for evaluating vision-centric reasoning beyond language.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。