用分步评估与策略优化，让小模型也能高效自主操作图形界面。
Building Autonomous GUI Navigation via Agentic-Q Estimation and Step-Wise Policy Optimization
- 自生成动作轨迹，用分步价值评估优化策略
- 小模型在导航任务中超越更大规模对手
- 适合想低成本部署智能界面助手的开发者
多模态大模型推动了图形界面自治代理的发展。然而在实际应用中，界面环境动态变化，导致数据收集和策略优化成本高昂。本文提出一种以多模态大模型为核心的框架，包含两项核心技术：一是代理Q值估计，用于生成分步价值，评估动作对任务完成的贡献；二是分步策略优化，利用自身生成的状态-动作轨迹进行强化学习，优化策略。该方法具有两个优势：（i）所有轨迹由策略自生成，数据采集成本可控；（ii）策略更新与环境解耦，保障优化稳定高效。实验表明，该框架使Ovis2.5-9B具备强大界面交互能力，在GUI导航与定位基准测试中表现优异，甚至超越参数量更大的竞争模型。
原文摘要 · Abstract (English)
Recent advances in Multimodal Large Language Models (MLLMs) have substantially driven the progress of autonomous agents for Graphical User Interface (GUI). Nevertheless, in real-world applications, GUI agents are often faced with non-stationary environments, leading to high computational costs for data curation and policy optimization. In this report, we introduce a novel MLLM-centered framework for GUI agents, which consists of two components: agentic-Q estimation and step-wise policy optimization. The former one aims to optimize a Q-model that can generate step-wise values to evaluate the contribution of a given action to task completion. The latter one takes step-wise samples from the state-action trajectory as inputs, and optimizes the policy via reinforcement learning with our agentic-Q model. It should be noticed that (i) all state-action trajectories are produced by the policy itself, so that the data collection costs are manageable; (ii) the policy update is decoupled from the environment, ensuring stable and efficient optimization. Empirical evaluations show that our framework endows Ovis2.5-9B with powerful GUI interaction capabilities, achieving remarkable performances on GUI navigation and grounding benchmarks and even surpassing contenders with larger scales.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。