用视觉反馈和强化学习提升网页生成质量，让代码更像真实网站。
WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning
- 引入多层级视觉反馈，通过截图和界面测试生成详细改进建议。
- 在WebGen-Bench上将Claude-3.5-Sonnet准确率从26.4%提升至51.9%。
- 提出步级强化学习方法，用视觉评分指导模型逐步优化生成过程。
基于大语言模型的智能体在代码生成任务中表现优异，但在依赖视觉效果与用户交互反馈的网页生成任务中，现有方法仅依赖简单代码执行验证，难以捕捉生成代码的真实质量。本文提出WebGen-Agent，通过整合视觉语言模型（VLM）生成的截图描述、界面测试反馈及量化评分，提供多层次视觉反馈，实现代码的迭代优化。结合回溯与最优选择机制，显著提升生成质量。进一步引入基于截图和界面测试评分的步级强化学习（Step-GRPO），以每一步的视觉评分作为奖励信号，增强模型推理能力。在WebGen-Bench数据集上，WebGen-Agent使Claude-3.5-Sonnet准确率由26.4%升至51.9%，外观评分从3.0增至3.9；对Qwen2.5-Coder-7B-Instruct，准确率从38.9%提升至45.4%，外观评分从3.4升至3.7，超越现有最先进系统。
原文摘要 · Abstract (English)
Agent systems powered by large language models (LLMs) have demonstrated impressive performance on repository-level code-generation tasks. However, for tasks such as website codebase generation, which depend heavily on visual effects and user-interaction feedback, current code agents rely only on simple code execution for feedback and verification. This approach fails to capture the actual quality of the generated code. In this paper, we propose WebGen-Agent, a novel website-generation agent that leverages comprehensive and multi-level visual feedback to iteratively generate and refine the website codebase. Detailed and expressive text descriptions and suggestions regarding the screenshots and GUI-agent testing of the websites are generated by a visual language model (VLM), together with scores that quantify their quality. The screenshot and GUI-agent scores are further integrated with a backtracking and select-best mechanism, enhancing the performance of the agent. Utilizing the accurate visual scores inherent in the WebGen-Agent workflow, we further introduce \textit{Step-GRPO with Screenshot and GUI-agent Feedback} to improve the ability of LLMs to act as the reasoning engine of WebGen-Agent. By using the screenshot and GUI-agent scores at each step as the reward in Step-GRPO, we provide a dense and reliable process supervision signal, which effectively improves the model's website-generation ability. On the WebGen-Bench dataset, WebGen-Agent increases the accuracy of Claude-3.5-Sonnet from 26.4% to 51.9% and its appearance score from 3.0 to 3.9, outperforming the previous state-of-the-art agent system. Additionally, our Step-GRPO training approach increases the accuracy of Qwen2.5-Coder-7B-Instruct from 38.9% to 45.4% and raises the appearance score from 3.4 to 3.7.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。