让开源视觉语言模型学会用代码和网页思考图像,性能超GPT-4o。
Visual Agentic Reinforcement Fine-Tuning
- 通过强化学习微调,让模型能调用工具浏览网页、写代码处理图像。
- 在多跳问答任务上提升29.3% F1,搜索与编码能力分别提升10.3%和18.6%。
- 适合想构建真实世界图像推理系统的开发者和研究者。
大型推理模型(如OpenAI的o3)正具备原生代理能力,可使用浏览器搜索、编写并执行代码以实现图像操作和图像思维。开源社区虽在纯语言代理能力方面取得进展,但涉及真正图像思维的多模态代理能力及其评测基准仍较少被探索。本文提出视觉代理强化微调(Visual-ARFT),使开源视觉语言模型(LVLMs)具备实时浏览网页获取信息、通过裁剪、旋转等技术编写代码处理和分析输入图像的能力。我们还构建了多模态代理工具评测集(MAT),包含两种设置:MAT-Search(搜索)与MAT-Coding(编程)。实验结果表明,Visual-ARFT在MAT-Coding上较基线提升+18.6% F1 / +13.0% EM,MAT-Search上提升+10.3% F1 / +8.7% EM,最终超越GPT-4o。在2Wiki和HotpotQA等多跳问答基准上,其F1和EM分别提升+29.3%和+25.9%,展现出强大泛化能力。结果表明,Visual-ARFT为构建鲁棒、通用的多模态智能体提供了可行路径。
原文摘要 · Abstract (English)
A key trend in Large Reasoning Models (e.g., OpenAI's o3) is the native agentic ability to use external tools such as web browsers for searching and writing/executing code for image manipulation to think with images. In the open-source research community, while significant progress has been made in language-only agentic abilities such as function calling and tool integration, the development of multi-modal agentic capabilities that involve truly thinking with images, and their corresponding benchmarks, are still less explored. This work highlights the effectiveness of Visual Agentic Reinforcement Fine-Tuning (Visual-ARFT) for enabling flexible and adaptive reasoning abilities for Large Vision-Language Models (LVLMs). With Visual-ARFT, open-source LVLMs gain the ability to browse websites for real-time information updates and write code to manipulate and analyze input images through cropping, rotation, and other image processing techniques. We also present a Multi-modal Agentic Tool Bench (MAT) with two settings (MAT-Search and MAT-Coding) designed to evaluate LVLMs' agentic search and coding abilities. Our experimental results demonstrate that Visual-ARFT outperforms its baseline by +18.6% F1 / +13.0% EM on MAT-Coding and +10.3% F1 / +8.7% EM on MAT-Search, ultimately surpassing GPT-4o. Visual-ARFT also achieves +29.3 F1% / +25.9% EM gains on existing multi-hop QA benchmarks such as 2Wiki and HotpotQA, demonstrating strong generalization capabilities. Our findings suggest that Visual-ARFT offers a promising path toward building robust and generalizable multimodal agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。