用设计师真实反馈提升UI生成模型质量
Improving User Interface Generation Models from Designer Feedback
- 通过评论、草图等设计习惯交互收集反馈
- 1500条标注数据使模型生成的UI质量显著提升
- 适合关注人机协同设计的AI研发者
尽管大型语言模型（LLM）经过海量数据训练，仍难以稳定生成优质用户界面。设计师反馈对提升UI生成效果至关重要；然而，现有基于评分或排序的强化学习人类反馈（RLHF）方法与设计师工作流程不匹配，且忽略了其用于批评和改进设计的丰富理由。本文研究了多种设计师提供反馈的方式，包括评论、草图和直接操作等熟悉交互。我们首先邀请21位设计师使用这些方式提供反馈，共生成1500条设计标注。随后利用该数据微调一系列LLM以生成更高质量的UI。最后通过人工评估发现，采用设计师对齐方法的模型优于传统排序反馈训练的模型及所有测试基线，包括GPT-5。
原文摘要 · Abstract (English)
Despite being trained on vast amounts of data, most LLMs are unable to reliably generate well-designed UIs. Designer feedback is essential to improving performance on UI generation; however, we find that existing RLHF methods based on ratings or rankings are not well-aligned with with designers' workflows and ignore the rich rationale used to critique and improve UI designs. In this paper, we investigate several approaches for designers to give feedback to UI generation models, using familiar interactions such as commenting, sketching and direct manipulation. We first perform an evaluation with 21 designers where they gave feedback using these interactions, which resulted in 1500 design annotations. We then use this data to finetune a series of LLMs to generate higher quality UIs. Finally, we evaluate these models with human judges, and we find that our designer-aligned approaches outperform models trained with traditional ranking feedback and all tested baselines, including GPT-5.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。