构建首个专业剪辑流程的GUI智能体评测基准，揭示当前模型在长序列任务中的局限性。
CutVerse: A Compositional GUI Agents Benchmark for Media Post-Production Editing
- 设计可组合的界面动作轨迹解析器，将屏幕录屏转为结构化操作序列
- 涵盖186个真实剪辑任务，跨7个专业软件，成功率仅36.0%
- 适合研究长程规划与多模态交互的AI agents开发者
尽管GUI智能体在网页导航和基础操作系统任务中取得进展，其在专业创意工作流中的能力仍鲜少被探索。为此，我们提出CutVerse，一个系统性评估自主GUI智能体在真实媒体后期制作环境中的基准。我们收集了涵盖7个专业应用（如Premiere Pro、Photoshop）的专家示范，覆盖186个复杂、长周期任务，基于真实编辑流程，涉及密集的多模态界面和紧密耦合的操作序列。为支持可扩展评估，我们开发了一种轻量级解析器，将原始屏幕录制和低层级交互日志转化为具有精确定位的结构化、组合式GUI操作轨迹。大量评估显示，现有智能体在真实媒体编辑任务上的任务成功率为36.0%，凸显了本基准所设复杂长周期后期制作工作流带来的挑战。尽管当前模型在空间定位、多模态对齐和协同操作执行方面表现良好，但在长周期可靠性与领域特定规划方面仍受限。
原文摘要 · Abstract (English)
While GUI agents have made significant progress in web navigation and basic operating system tasks, their capabilities in professional creative workflows remain largely underexplored. To bridge this gap, we introduce Cutverse, a benchmark designed to systematically evaluate autonomous GUI agents in realistic media post-production environments. We curate expert demonstrations across 7 professional applications (e.g., Premiere Pro, Photoshop), covering 186 complex, long-horizon tasks grounded in authentic editing workflows, involving dense multimodal interfaces and tightly coupled interaction sequences. To support scalable evaluation, we develop a lightweight parser that transforms raw screen recordings and low-level interaction logs into structured, compositional GUI action trajectories with precise grounding. Extensive evaluations reveal that existing agents achieve only 36.0\% task success on realistic media editing tasks, underscoring the challenges posed by complex, long-horizon media post-production workflows in our benchmark.While current models demonstrate promising spatial grounding, multimodal alignment, and coordinated action execution, they remain limited in long-horizon reliability and domain-specific planning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。