让网页智能体学会预判操作后果，避免重复下单等致命错误。
Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation
- 用自然语言描述状态变化，构建可预测的环境动态模型
- 在WebArena和Mind2Web上实现零训练成本的策略优化
- 适合需要高可靠性长序列决策的自动化场景
大型语言模型（LLMs）在构建自主智能体方面备受关注，但现有基于LLM的网页智能体在长周期任务中的表现仍不理想，常出现重复购买不可退票等不可逆错误。相比之下，人类能通过预判行为后果（如损失金钱）来规避此类错误，这依赖于对环境的“世界模型”。本文首先通过初步分析证实当前主流大模型（如GPT-4o、Claude-3.5-Sonnet）缺乏世界模型能力。为此，我们提出一种增强世界模型的网页智能体（WMA），通过模拟动作后果提升决策质量。针对训练大模型作为世界模型时面临的挑战（如观察中重复内容、长HTML输入），我们设计了以转换为核心的观察抽象方法，仅预测两时刻间关键状态差异的自由文本描述。在WebArena与Mind2Web上的实验表明，该世界模型可在不额外训练的情况下显著改进智能体策略选择，并在成本与时间效率上优于近期基于树搜索的方法。
原文摘要 · Abstract (English)
Large language models (LLMs) have recently gained much attention in building autonomous agents. However, the performance of current LLM-based web agents in long-horizon tasks is far from optimal, often yielding errors such as repeatedly buying a non-refundable flight ticket. By contrast, humans can avoid such an irreversible mistake, as we have an awareness of the potential outcomes (e.g., losing money) of our actions, also known as the "world model". Motivated by this, our study first starts with preliminary analyses, confirming the absence of world models in current LLMs (e.g., GPT-4o, Claude-3.5-Sonnet, etc.). Then, we present a World-model-augmented (WMA) web agent, which simulates the outcomes of its actions for better decision-making. To overcome the challenges in training LLMs as world models predicting next observations, such as repeated elements across observations and long HTML inputs, we propose a transition-focused observation abstraction, where the prediction objectives are free-form natural language descriptions exclusively highlighting important state differences between time steps. Experiments on WebArena and Mind2Web show that our world models improve agents' policy selection without training and demonstrate our agents' cost- and time-efficiency compared to recent tree-search-based agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。