arXiv:2604.08516cs.CV2026-04被引 17

开源网页智能体与数据集,让网页自动化更透明可复现。

MolmoWeb: Open Visual Web Agent and Open Data for the Open Web

  • 构建了超10万条合成+3万条真人操作的网页交互数据集
  • 4B/8B模型在多个基准上超越同类开源模型,8B版胜过闭源大模型
  • 支持无代码、无网页结构信息的视觉指令执行,适合研究者快速实验

网页智能体能代表用户自主完成网络任务,但当前顶尖系统依赖封闭模型和训练数据,阻碍了科学进展。为此,我们推出(1)MolmoWebMix——包含10万条以上合成任务轨迹、3万+真人操作记录及网页界面感知数据的大规模混合数据集;(2)MolmoWeb系列全开源多模态网页智能体。MolmoWeb通过视觉-语言指令生成下一步浏览器动作,无需访问HTML、无障碍树或专用API。4B与8B版本在WebVoyager、Online-Mind2Web和DeepShop等基准上表现领先,8B版甚至超越基于更大闭源模型(如GPT-4o)的set-of-marks代理。通过测试时并行推演与best-of-N选择,其在WebVoyager和Online-Mind2Web上的pass@4分别达到94.7%和60.5%(对比pass@1为78.2%和35.3%)。我们将公开模型检查点、训练数据、代码与统一评估工具,推动开放研究。

原文摘要 · Abstract (English)

Web agents--autonomous systems that navigate and execute tasks on the web on behalf of users--have the potential to transform how people interact with the digital world. However, the most capable web agents today rely on proprietary models with undisclosed training data and recipes, limiting scientific understanding, reproducibility, and community-driven progress. We believe agents for the open web should be built in the open. To this end, we introduce (1) MolmoWebMix, a large and diverse mixture of browser task demonstrations and web-GUI perception data and (2) MolmoWeb, a family of fully open multimodal web agents. Specifically, MolmoWebMix combines over 100K synthetic task trajectories from multiple complementary generation pipelines with 30K+ human demonstrations, atomic web-skill trajectories, and GUI perception data, including referring expression grounding and screenshot question answering. MolmoWeb agents operate as instruction-conditioned visual-language action policies: given a task instruction and a webpage screenshot, they predict the next browser action, requiring no access to HTML, accessibility trees, or specialized APIs. Available in 4B and 8B size, on browser-use benchmarks like WebVoyager, Online-Mind2Web, and DeepShop, MolmoWeb agents achieve state-of-the-art results outperforming similar scale open-weight-only models such as Fara-7B, UI-Tars-1.5-7B, and Holo1-7B. MolmoWeb-8B also surpasses set-of-marks (SoM) agents built on much larger closed frontier models like GPT-4o. We further demonstrate consistent gains through test-time scaling via parallel rollouts with best-of-N selection, achieving 94.7% and 60.5% pass@4 (compared to 78.2% and 35.3% pass@1) on WebVoyager and Online-Mind2Web respectively. We will release model checkpoints, training data, code, and a unified evaluation harness to enable reproducibility and accelerate open research on web agents.

网页智能体开源数据多模态自动化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。