arXiv:2603.09652cs.AI2026-03被引 5

首个评估大模型生成交互式网页应用的基准,填补了动态交互评测空白。

MiniAppBench: Evaluating the Shift from Text to Interactive HTML Responses in LLM-Powered Assistants

  • 构建涵盖6个领域的500个真实任务,聚焦原则驱动的交互应用生成
  • 现有大模型在生成高质量交互应用上仍存在明显不足
  • 提出自动化评估框架,模拟人类探索测试,适配开放性交互场景

随着大语言模型在代码生成方面的快速发展,人机交互正从静态文本响应转向动态、交互式的基于HTML的应用程序,我们称之为MiniApp。这些应用不仅要求模型渲染可视化界面,还需构建符合现实逻辑的定制化交互逻辑。然而,现有基准主要关注算法正确性或静态布局重建,无法捕捉这一新范式所需的能力。为此,我们提出MiniAppBench,首个全面评估原则驱动型交互应用生成的基准。数据源自一个产生超1000万次生成的真实应用,提炼出500个任务,覆盖游戏、科学、工具等六个领域。针对开放式交互中无单一真值的问题,我们提出MiniAppEval,一种基于浏览器自动化的代理评估框架。该框架通过类人探索式测试,系统评估应用在意图、静态与动态三个维度的表现。实验表明,当前大模型在生成高质量MiniApp方面仍面临显著挑战,而MiniAppEval与人类判断高度一致,为后续研究提供了可靠标准。

原文摘要 · Abstract (English)

With the rapid advancement of Large Language Models (LLMs) in code generation, human-AI interaction is evolving from static text responses to dynamic, interactive HTML-based applications, which we term MiniApps. These applications require models to not only render visual interfaces but also construct customized interaction logic that adheres to real-world principles. However, existing benchmarks primarily focus on algorithmic correctness or static layout reconstruction, failing to capture the capabilities required for this new paradigm. To address this gap, we introduce MiniAppBench, the first comprehensive benchmark designed to evaluate principle-driven, interactive application generation. Sourced from a real-world application with 10M+ generations, MiniAppBench distills 500 tasks across six domains (e.g., Games, Science, and Tools). Furthermore, to tackle the challenge of evaluating open-ended interactions where no single ground truth exists, we propose MiniAppEval, an agentic evaluation framework. Leveraging browser automation, it performs human-like exploratory testing to systematically assess applications across three dimensions: Intention, Static, and Dynamic. Our experiments reveal that current LLMs still face significant challenges in generating high-quality MiniApps, while MiniAppEval demonstrates high alignment with human judgment, establishing a reliable standard for future research. Our homepage is available in miniappbench.github.io.

大模型评估交互应用自动化测试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。