用视觉修复规则引导界面生成，让代码自进化更稳定。
Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation
- 引入视觉修复规则作为结构化反馈，指导代码修改方向。
- 在6个视觉语言模型上提升生成稳定性，最佳轮次准确率显著提高。
- 适合需要高精度界面生成的开发者和自动化工具设计者。
大型视觉-语言模型在界面到代码生成方面进展显著，但其测试时的自我演化仍不稳定。我们首次识别出根本障碍——视觉修复耦合：局部代码修改可能通过布局、样式和组件依赖传播，修正一处视觉不一致的同时破坏先前正确的区域。为此，我们提出RubSE框架，利用规则（rubrics）表示视觉反馈，形成结构化的视觉修复上下文。每轮优化中，RubSE生成带类型的候选规则，优先选择修复目标，并将历史选中的规则存入记忆，从而引导每次修改聚焦于特定修复，避免重复或过度泛化改动。在六个视觉语言模型和三个界面到代码基准上的评估表明，RubSE显著优于朴素自进化方法，在最终轮和最佳轮设置下表现更优，实现更稳定的优化轨迹与更高的性能上限。进一步分析显示，RubSE通过增强对严重视觉退化的恢复能力，缓解了轨迹坍缩问题；且更强的规则生成器能将有效的视觉修复引导能力迁移至较弱的代码改进器。
原文摘要 · Abstract (English)
Large vision-language models have shown strong progress in UI-to-code generation, yet their test-time self-evolution remains unstable. We first identify a fundamental obstacle, termed visual repair coupling: a local code edit may propagate through layout, style, and component dependencies, correcting one visual mismatch while degrading regions that were previously faithful. To address this issue, we present RubSE, a Rubric-guided Self-Evolution framework that uses rubrics to represent visual feedback as a structured visual-repair context. At each refinement round, RubSE generates typed candidate rubrics, selects one prioritized repair target, and stores previously selected rubrics as history, thereby steering each revision toward a well-scoped visual repair while discouraging repeated or over-broad changes. Evaluations across six VLMs and three UI-to-code benchmarks demonstrate that RubSE substantially outperforms naïve self-evolution in final-round and best-round settings, achieving more stable refinement trajectories and a higher trajectory-level performance ceiling. Further analysis shows that RubSE mitigates trajectory collapse by improving recovery from severe visual regressions, and that stronger rubric generators can transfer effective visual-repair guidance to weaker code improvers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。