测试大模型主动求助能力，发现多数模型不会主动请求帮助。
ProactiveBench: Benchmarking Proactiveness in Multimodal Large Language Models
- 构建多任务基准测试，评估模型在遮挡识别等场景中主动请求帮助的能力。
- 22个模型普遍缺乏主动性，且模型越大越不主动，提示也效果有限。
- 通过强化学习微调可有效提升主动性，适用于未见场景。
有效的协作始于知道何时寻求帮助。例如，当试图识别被遮挡的物体时，人类会请求他人移除障碍。多模态大语言模型能否表现出类似的“主动性”，通过请求简单的用户干预来协助？为探究此问题，我们提出了ProactiveBench，一个基于七个重构数据集的基准测试，用于评估不同任务（如识别遮挡物体、提升图像质量、解读粗糙草图）中的主动性。我们在22个MLLMs上进行评估，发现：(i) 它们普遍缺乏主动性；(ii) 主动性与模型容量无关；(iii) “提示”仅带来微弱提升。令人意外的是，对话历史和上下文学习反而引入负面偏见，阻碍表现。最后，我们探索了一种基于强化学习的简单微调策略，结果表明主动性可以被学习，并能泛化到未见场景。我们公开发布ProactiveBench，作为构建主动式多模态模型的第一步。
原文摘要 · Abstract (English)
Effective collaboration begins with knowing when to ask for help. For example, when trying to identify an occluded object, a human would ask someone to remove the obstruction. Can MLLMs exhibit a similar "proactive" behavior by requesting simple user interventions? To investigate this, we introduce ProactiveBench, a benchmark built from seven repurposed datasets that tests proactiveness across different tasks such as recognizing occluded objects, enhancing image quality, and interpreting coarse sketches. We evaluate 22 MLLMs on ProactiveBench, showing that (i) they generally lack proactiveness; (ii) proactiveness does not correlate with model capacity; (iii) "hinting" at proactiveness yields only marginal gains. Surprisingly, we found that conversation histories and in-context learning introduce negative biases, hindering performance. Finally, we explore a simple fine-tuning strategy based on reinforcement learning: its results suggest that proactiveness can be learned, even generalizing to unseen scenarios. We publicly release ProactiveBench as a first step toward building proactive multimodal models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。