发现生成式UI工具的解释理由与实际界面不符，存在‘设计表演’现象。
Design Theater: Evaluating the Gap Between User-Facing Design Reasoning and Implementation in Generative UI Tools
- 通过24项任务构建评测基准，检验设计理由是否真实落地。
- 平均超25%的设计理由未在界面中实现，功能需求失败率高达34%。
- 适合关注AI生成工具可信度与评估方法的研究者与开发者。
生成式UI工具承诺通过自然语言描述生成完整界面，并附上用户可见的设计理由，解释布局、可访问性及设计决策。然而，这些理由是否真实反映在生成界面中尚不明确。我们提出“设计表演”概念：看似合理且自信的设计理由与实际实现脱节。为此，我们构建了一个包含24项涵盖结构、样式和功能设计要求的基准，评估五款生成式工具产生的120个界面。结果显示，平均超过25%的用户可见设计理由未在界面中实现，功能类要求的实现失败率更高，达34%。工具仅识别出提示中约一半的用户体验原则（均值=0.54），其中四款工具仅实现6%或更少的功能性原则。我们还测量了不同工具间的界面相似性，发现视觉外观和布局组织趋于收敛，但配色选择差异较大。本研究贡献包括：1）提出“设计表演”概念；2）提供评测基准与三项量化指标；3）系统评估结果。讨论了对生成式UI工具设计与评估的启示。
原文摘要 · Abstract (English)
Generative UI tools promise to democratize UI design by turning natural language descriptions into complete interfaces. Alongside the interface, these tools generate user-facing design rationales that explain their layout, accessibility, and design choices. However, it remains unclear whether these stated rationales are actually reflected in the interfaces they produce. We call this disconnect ``Design Theater'': plausible and confident design rationales that have little relationship to the actual implementation. To study this phenomenon, we introduce a benchmark and three metrics for measuring Design Theater. The benchmark includes 24 UI generation tasks spanning structural, styling, and functional design requirements. Using this benchmark, we evaluate 120 interfaces created by five generative UI tools. On average, over 25\% of user-facing design rationales are not implemented in the generated interface, and the implementation failure increases to 34\% for functional requirements. Tools recognize roughly half of the UX principles embedded in prompts (mean = 0.54), with four of five tools implementing 6\% or fewer functional principles. We also measure interface similarity across tools and find convergence in visual appearance and layout organization, with greater variation in color choices. Overall, we contribute: 1) the concept of Design Theater; 2) a benchmark with metrics for assessing whether the stated reasoning of generative UI tools is reflected in their implementations; 3) and findings from a systematic evaluation of these tools. We discuss what these findings mean for the design and evaluation of generative UI tools.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。