用通用工具打造的智能体,跨领域解题能力更强。
Coding Agents with Multimodal Browsing are Generalist Problem Solvers
- 仅用代码编辑、搜索、多模态浏览等通用工具构建
- 在三大测试集上表现优于或媲美顶尖专用智能体
- 为通用智能体研究提供新基准,适合多任务研究者
现代人类劳动依赖专业分工,AI智能体也常针对特定领域(如编程、网页导航)定制。但这类智能体难以泛化。本文提出通用智能体OpenHands-Versa,仅使用代码编辑与执行、网络搜索、多模态网页浏览和文件访问四项通用工具,在三个挑战性基准(SWE-Bench Multimodal、GAIA、The Agent Company)上均取得领先或相当成绩,分别实现9.1、1.3、9.1个百分点的绝对性能提升。结果表明,无需高度定制化设计即可实现强泛化能力。同时发现现有顶尖多智能体系统无法跨域推广。本工作验证了通用智能体的可行性,并确立OpenHands-Versa为未来研究的重要基线。
原文摘要 · Abstract (English)
Modern human labor is characterized by specialization; we train for years and develop particular tools that allow us to perform well across a variety of tasks. In addition, AI agents have been specialized for domains such as software engineering, web navigation, and workflow automation. However, this results in agents that are good for one thing but fail to generalize beyond their intended scope. One reason for this is that agent developers provide a highly specialized set of tools or make architectural decisions optimized for a specific use case or benchmark. In this work, we ask the question: what is the minimal set of general tools that can be used to achieve high performance across a diverse set of tasks? Our answer is OpenHands-Versa, a generalist agent built with a modest number of general tools: code editing and execution, web search, as well as multimodal web browsing and file access. Importantly, OpenHands-Versa demonstrates superior or competitive performance over leading specialized agents across three diverse and challenging benchmarks: SWE-Bench Multimodal, GAIA, and The Agent Company, outperforming the best-performing previously published results with absolute improvements in success rate of 9.1, 1.3, and 9.1 points respectively. Further, we show how existing state-of-the-art multi-agent systems fail to generalize beyond their target domains. These results demonstrate the feasibility of developing a generalist agent to solve diverse tasks and establish OpenHands-Versa as a strong baseline for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。