用语义中间层让AI生成界面更可控、可理解。
Bridging Gulfs in UI Generation through Semantic Guidance
- 将设计语义作为人意与AI输出的中间表示
- 用户可可视化语义关系并追溯其在界面中的体现
- 提升用户对设计意图和结果的掌控感
尽管生成式AI能根据文本提示生成高保真界面，但用户难以准确表达设计意图，也难以评估或改进生成结果，导致执行与评价之间的鸿沟。我们通过主题分析界面提示指南，识别出关键设计语义，并发现其具有层级性和相互依赖性。基于此，我们开发了一套系统，使用户能够指定语义、可视化语义关系，并提取语义在生成界面中的具体体现。通过将语义作为人意与AI输出之间的中间表示，该系统同时弥合了执行与评价的鸿沟，使需求更明确、结果更可解释。对比用户研究表明，该方法提升了用户对意图表达和结果解读的感知控制力，促进了更可预测的迭代优化。本工作展示了显式语义表示如何实现AI驱动界面设计中系统化且可解释的设计探索。
原文摘要 · Abstract (English)
While generative AI enables high-fidelity UI generation from text prompts, users struggle to articulate design intent and evaluate or refine results-creating gulfs of execution and evaluation. To understand the information needed for UI generation, we conducted a thematic analysis of UI prompting guidelines, identifying key design semantics and discovering that they are hierarchical and interdependent. Leveraging these findings, we developed a system that enables users to specify semantics, visualize relationships, and extract how semantics are reflected in generated UIs. By making semantics serve as an intermediate representation between human intent and AI output, our system bridges both gulfs by making requirements explicit and outcomes interpretable. A comparative user study suggests that our approach enhances users' perceived control over intent expression and outcome interpretation, and facilitates more predictable iterative refinement. Our work demonstrates how explicit semantic representation enables systematic and explainable exploration of design possibilities in AI-driven UI design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。