打造可实时协作的AI编程实验平台，研究开发者如何用AI做网页开发。
VibeJam: A User Study Platform for Web Development with Agents
- 基于浏览器构建交互式开发平台，支持代码编辑、差分审查与实时预览
- 55个游戏化任务中，初学者用该平台生成的网站质量优于纯代理生成结果
- 开源工具，适合研究人机协作编程，尤其关注网页开发场景
随着AI在编程中的应用日益智能化，用户通过提示大语言模型直接编辑代码并审查修改，这一趋势在网页开发领域尤为明显。然而，现有NLP研究多依赖离线评估，缺乏对真实编程者使用行为的洞察。为此，我们发布VibeJam——一个基于浏览器的用户研究平台，支持用户与AI代理协同开发网站。平台默认集成开源Aider代理，提供差分审查、对话、计划模式及实时网站预览功能，以贴近实际应用场景。在一项包含55个已发布的游戏化网站创作任务的试点研究中，五位有经验的AI程序员评价系统有趣、易用且接近商业工具；13名初级学生使用该平台所创建的网站质量高于相同任务下纯代理生成的结果。我们已开源VibeJam，旨在推动后续扩展，并支持对编码代理如何助力用户的深入研究。
原文摘要 · Abstract (English)
Programming with AI is increasingly agentic, users prompt LLMs to directly edit their code and review the changes, with adoption growing especially for web development tasks. Despite this growth, most NLP work uses offline evaluation and lacks support for online studies, losing insights into how programmers truly use coding agents. We release VibeJam, a browser-based user study platform for users to collaborate with AI agents to develop websites. VibeJam enables agent customization and uses the open-source Aider agent by default, and to mirror downstream use, we add diff review, chat and plan modes, and live website previews. In a pilot study with 55 released, game-based website creation tasks, five experienced AI programmers rate our system as fun, simple, and resembling commercial tools, while 13 junior students use VibeJam to make websites of higher quality than agents in the same task. We open-source VibeJam to spur extensions and support studies on how coding agents can help users.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。