首个面向多屏移动端应用生成的全流程评估基准
Looks Right, Works Right: A Project-Level Benchmark for Multi-Screen Mobile App Generation
- 构建真实应用的多页代码项目，支持跨页导航与组件复用
- 现有模型可生成可编译代码但交互导航仍不可靠
- 适合评估大模型在真实开发场景中的端到端能力
近期多模态大模型可将视觉设计直接转为可执行代码，但真实移动端产品需多截图构建具备共享组件和可运行导航的完整代码库。现有设计到代码的评测基准存在三大局限：仅关注单页生成、无法评估跨页导航、未衡量项目整体可维护性。为此，我们提出 MobileForge——首个面向项目级多屏移动端应用生成的基准，包含真实移动应用、人工审核的屏幕数据、结构化页面关系标注及导航测试规范。MobileForge 支持构建、导航、视觉保真度、代码可维护性和效率五维度评估。我们还提出状态隔离的导航测试方法以避免失败传播，并采用锚点参考的列表式视觉评估协议提升评判可靠性。在六种前沿多模态大模型上进行端到端测试发现，当前模型虽能生成可编译且能到达正确页面的应用，但交互导航仍不可靠，视觉保真度和可维护性仍有显著差距。相关基准与资源已开源：https://github.com/anoa12159-hue/mobileforge_eval。
原文摘要 · Abstract (English)
Recent multimodal large language models can convert visual designs directly into executable code, but real mobile products require multiple screenshots to become a buildable codebase with shared components and working navigation. This project-level setting exposes three limits of existing design-to-code benchmarks: they focus on single-page generation rather than complete codebases, cannot evaluate cross-page navigation, and do not measure project-wide maintainability. We introduce MobileForge, the first benchmark for project-level multi-screen mobile app generation, comprising real mobile apps, human-reviewed screens, structured page-relationship annotations, and navigation test specifications. MobileForge supports five-axis evaluation of build, navigation, visual fidelity, code maintainability, and efficiency. We also propose state-isolated navigation testing to avoid cascading failures in navigation evaluation and an anchor-referenced list-wise visual evaluation protocol to improve visual-judge reliability. Across end-to-end runs on six frontier multimodal LLMs, current models can build mobile-app projects that compile and reach the correct pages, but interactive navigation remains unreliable and visual fidelity and maintainability still lag. The benchmark and supporting materials are available at https://github.com/anoa12159-hue/mobileforge_eval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。