构建可验证的合成网页环境，让智能体训练更可靠。
Training Needs Trustworthy Worlds: Verified Synthetic Web Environments for Agent Learning
- 用结构化框架生成可执行、可审计的网页环境
- 任务失败率从48.6%降至94.8%，提升可执行任务比例
- 适合想训练高效网页智能体的研究者与开发者
网页智能体有望自动化复杂数字流程，但其训练受限于表面看似合理却存在失效链接、状态不一致或不可行任务的合成环境。我们通过构建可执行、可审计且基于后端状态的合成网页环境，填补了大规模环境生成与可信智能体学习之间的差距。框架将每个生成网站表示为页面、导航链接、数据库记录、状态变更标记和任务约束的结构化架构，并在策略训练前验证并修复结构、语义、一致性及可行性缺陷。交互过程中，普通界面跳转确定性执行，而持久后端更新仅通过经验证的状态变更标记触发，从而实现基于已验证任务进度谓词的密集奖励。在覆盖六个领域的500个合成环境中，该方法将任务阻塞缺陷减少，可行任务率从48.6%提升至94.8%，生成更强的PPO策略，并在WebArena、WebShop和MiniWoB++上实现良好迁移，评估时无需大模型调用。结果表明，经过验证的合成环境可作为紧凑网页智能体的可扩展、可靠训练基础，推动网页智能体学习从表面合理性转向可执行、状态对齐的监督。
原文摘要 · Abstract (English)
Web agents promise to automate complex digital workflows, but their training remains limited by synthetic environments that look plausible while hiding broken links, inconsistent states, or infeasible tasks. We address the gap between scalable environment generation and trustworthy agent learning by constructing synthetic web environments that are executable, auditable, and grounded in backend state. Our framework represents each generated website as a structured scaffold of pages, navigation links, database records, state-change markers, and task constraints, then verifies and repairs structural, semantic, consistency, and feasibility defects before policy training. During interaction, ordinary UI transitions are executed deterministically, while persistent backend updates are invoked only through validated state-change markers, enabling dense rewards compiled from verified task-progress predicates. Across 500 synthetic environments spanning six domains, our method reduces task-blocking defects and improves feasible-task rate from 48.6% to 94.8%, while producing stronger PPO policies and improving transfer to WebArena, WebShop, and MiniWoB++ without LLM calls at evaluation time. These results show that verified synthetic environments can serve as a scalable and reliable training substrate for compact web agents, shifting synthetic webagent learning from surface-level plausibility toward executable, state-grounded supervision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。