arXiv:2602.12892cs.CVcs.AI2026-02

提出RADAR框架,精准诊断多模态大模型预训练中的感知与推理能力差异。

RADAR: Revealing Asymmetric Development of Abilities in MLLM Pre-training

  • 设计软判别得分,无需微调即可追踪模型能力演变
  • 构建1.5万+样本多模态混合基准,实现零样本全面评估
  • 发现模型在数据量、规模和策略下感知与推理发展不均衡

预训练多模态大语言模型(MLLM)凭借其内在的感知与推理能力,为后续训练提供知识基础。然而,缺乏高效的评估框架阻碍了对性能瓶颈的诊断。现有评估主要依赖监督微调后的测试,带来额外训练与自回归解码开销;通用预训练指标无法解耦量化感知与推理能力;现有评测基准在规模或目标上存在局限。为此,我们提出RADAR——一种用于揭示多模态大模型预训练中能力不对称发展的高效能力中心评估框架。RADAR包含两个核心组件:(1) 软判别得分,基于模型对正确答案与干扰项的偏好细微梯度,实现无需微调的能力追踪;(2) 多模态混合基准,一个包含1.5万+样本的新基准,通过整合权威数据集并精心收集新数据,在零样本条件下全面评估感知与推理能力,拓展评估范围并填补当前基准关键空白。借助RADAR,我们系统揭示了不同因素(如数据量、模型规模、预训练策略)下感知与推理能力发展的不对称性。结果表明,需采用解耦视角识别预训练瓶颈,从而实施针对性优化以更高效推进MLLM发展。代码已开源:https://github.com/Nieysh/RADAR。

原文摘要 · Abstract (English)

Pre-trained Multi-modal Large Language Models (MLLMs) provide a knowledge-rich foundation for post-training by leveraging their inherent perception and reasoning capabilities to solve complex tasks. However, the lack of an efficient evaluation framework impedes the diagnosis of their performance bottlenecks. Current evaluation primarily relies on testing after supervised fine-tuning, which introduces laborious additional training and autoregressive decoding costs. Meanwhile, common pre-training metrics cannot quantify a model's perception and reasoning abilities in a disentangled manner. Furthermore, existing evaluation benchmarks are typically limited in scale or misaligned with pre-training objectives. Thus, we propose RADAR, an efficient ability-centric evaluation framework for Revealing Asymmetric Development of Abilities in MLLM pRe-training. RADAR involves two key components: (1) Soft Discrimination Score, a novel metric for robustly tracking ability development without fine-tuning, based on quantifying nuanced gradations of the model preference for the correct answer over distractors; and (2) Multi-Modal Mixture Benchmark, a new 15K+ sample benchmark for comprehensively evaluating pre-trained MLLMs' perception and reasoning abilities in a 0-shot manner, where we unify authoritative benchmark datasets and carefully collect new datasets, extending the evaluation scope and addressing the critical gaps in current benchmarks. With RADAR, we comprehensively reveal the asymmetric development of perceptual and reasoning capabilities in pretrained MLLMs across diverse factors, including data volume, model size, and pretraining strategy. Our RADAR underscores the need for a decomposed perspective on pre-training ability bottlenecks, informing targeted interventions to advance MLLMs efficiently. Our code is publicly available at https://github.com/Nieysh/RADAR.

多模态评估框架能力解耦预训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。