arXiv:2608.06474cs.AI2026-08

用自演化程序化评分器训练大模型,让网页生成更准确可靠。

WebGrader: Training LLMs for Web Development with Self-Evolving Programmatic Grader

论文配图:WebGrader: Training LLMs for Web Development with Self-Evolving Programmatic Grader
图 1 · 摘自论文原文
  • 自动生成交互流程并以可执行契约形式评分
  • 在真实浏览器中运行,收集多维度证据后判定成功
  • 适合想提升网页生成准确率的研究者和开发者

大型语言模型正从自然语言描述生成完整网站,强化学习成为弥补其功能差距的核心方法。但该训练受制于奖励设计瓶颈:人工编写的浏览器脚本虽可执行却成本高昂,而视觉语言模型与GUI代理评分器虽可扩展,却可能在观察到决定性状态前做出判断。本文提出WebGrader,一种自演化程序化评分器,能自主从每个网站请求中推导出所需交互流程,将其表示为可执行的Flow Contract,并以执行结果作为强化学习奖励。WebGrader在真实浏览器中实例化生成项目,将目标动作与源代码及实时DOM对齐,沿同一浏览器轨迹收集视觉、DOM、响应及持久状态证据。通过残差驱动的离线循环,发现可复用的验证技能,在独立验证页面上筛选并冻结优选技能图谱后进行策略训练。通过分离测试规划、动作定位、证据采集与语义判断,WebGrader仅在观察到请求转移后才给出通过判定。在WebGen-Bench上,WebGrader训练出的8B模型达到52.01%的功能成功率,比匹配的外观加脚本奖励高出7.88点,优于o4-mini和DeepSeek-v4-flash。在WG-core-250上,模型获得44.953分全分,超越Qwen3-Coder-480B。

原文摘要 · Abstract (English)

Large language models increasingly generate complete websites from natural-language descriptions, and reinforcement learning has become a central approach to closing their remaining functional gap. This training regime is bottlenecked by reward design. Hand-authored browser scripts are executable yet costly to write for open-ended requirements, while VLM and GUI-agent graders scale but may issue verdicts before observing the decisive state. We propose WebGrader, a self-evolving programmatic grader that autonomously derives the required interaction flows from each website request, represents each flow as an executable Flow Contract, and uses its execution outcome as an RL reward. WebGrader materializes the generated project in a live browser, grounds target actions against the source code and live DOM, and collects visual, DOM, response, and persistent-state evidence along the same browser trajectory. A residual-driven offline loop then discovers reusable verifier skills, screens them on disjoint validation pages, and freezes the promoted skill graph before policy training. By separating test planning, action grounding, evidence collection, and semantic judgment, WebGrader issues a Pass verdict only after observing the requested transition. On WebGen-Bench, WebGrader trains an 8B policy to a 52.01% functional success rate, outperforming a matched appearance-plus-script reward by 7.88 points and surpassing o4-mini and DeepSeek-v4-flash. On WG-core-250, the policy reaches a Full Score of 44.953 and surpasses Qwen3-Coder-480B.

大模型网页生成强化学习自动评测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。