1D-Bench评估模型在真实电商场景中通过迭代编辑生成可执行前端代码的能力。
1D-Bench: A Benchmark for Iterative UI Code Generation with Visual Feedback in Real-World
- 基于真实电商流程构建,使用中间表示与参考渲染对比评测
- 迭代编辑使渲染成功率提升,视觉相似度也有所改善
- 适合研究设计转代码、迭代修复与强化学习编辑的学者
设计转代码将高保真UI设计转化为可执行的前端实现,但因数据集、工具链和评估协议不一致,进展难以比较。我们提出1D-Bench,一个基于真实电商工作流的基准测试,每个任务提供参考渲染和可能含提取错误的中间表示。1D代表一天内完成任务,模型以中间表示为结构线索,以参考渲染为评估目标,测试对中间表示缺陷的鲁棒性而非字面匹配。1D-Bench要求在固定工具链下生成具备显式组件层级的可执行React代码库,并采用多轮迭代编辑设置,利用执行反馈进行组件级修改。在商业及开源多模态模型上的实验表明,迭代编辑通常能提高最终性能,提升渲染成功率并改善视觉相似度。我们还开展了合成修复轨迹后训练与基于强化学习的编辑试点研究,观察到收益有限且不稳定,可能源于稀疏终端奖励和高方差文件级更新。本研究使用的数据与脚本已匿名发布于https://anonymous.4open.science/r/d2c-benchmark-A9C4/。
原文摘要 · Abstract (English)
Design-to-code translates high-fidelity UI designs into executable front-end implementations, but progress remains hard to compare due to inconsistent datasets, toolchains, and evaluation protocols. We introduce 1D-Bench, a benchmark grounded in real e-commerce workflows, where each instance provides a reference rendering and an exported intermediate representation that may contain extraction errors. 1D is short for one day, representing the efficient completion of design-to-code tasks in less than one day. Models take both as input, using the intermediate representation as structural cues while being evaluated against the reference rendering, which tests robustness to intermediate representation defects rather than literal adherence. 1D-Bench requires generating an executable React codebase under a fixed toolchain with an explicit component hierarchy, and defines a multi-round setting in which models iteratively apply component-level edits using execution feedback. Experiments on commercial and open-weight multimodal models show that iterative editing generally improves final performance by increasing rendering success and often improving visual similarity. We further conduct a pilot study on post-training with synthetic repair trajectories and reinforcement learning based editing, and observe limited and unstable gains that may stem from sparse terminal rewards and high-variance file-level updates. The data and scripts used in this study are available in an anonymized repository at https://anonymous.4open.science/r/d2c-benchmark-A9C4/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。