动态设定里程碑奖励，解决移动端智能体长程任务中的奖励稀疏问题。
Adaptive Milestone Reward for GUI Agents
- 基于成功探索自动提取动态里程碑，构建可验证的自适应奖励机制。
- 在AndroidWorld上使成功率绝对提升超10%，且跨模型和环境表现稳定。
- 适合需要长程决策与可靠奖励信号的GUI自动化任务研究者。
强化学习已成为训练移动端GUI智能体的主流方法，但在长时程任务中面临时间信用分配难题。核心挑战在于奖励保真度与密度的权衡：结果奖励虽保真度高但信号稀疏，过程奖励虽密集却易受偏差和奖励劫持影响。为此，我们提出自适应里程碑奖励（ADMIRE）机制，通过从成功探索中动态提炼里程碑，构建可验证的自适应奖励系统。关键在于引入非对称信用分配策略，对成功轨迹去噪并为失败轨迹提供引导。大量实验表明，ADMIRE在不同基础模型上于AndroidWorld任务中持续实现超过10%的绝对成功率提升，且具备强泛化能力，在多种RL算法及异构环境（如网页导航、具身任务）中均表现优异。
原文摘要 · Abstract (English)
Reinforcement Learning (RL) has emerged as a mainstream paradigm for training Mobile GUI Agents, yet it struggles with the temporal credit assignment problem inherent in long-horizon tasks. A primary challenge lies in the trade-off between reward fidelity and density: outcome reward offers high fidelity but suffers from signal sparsity, while process reward provides dense supervision but remains prone to bias and reward hacking. To resolve this conflict, we propose the Adaptive Milestone Reward (ADMIRE) mechanism. ADMIRE constructs a verifiable, adaptive reward system by anchoring trajectory to milestones, which are dynamically distilled from successful explorations. Crucially, ADMIRE integrates an asymmetric credit assignment strategy that denoises successful trajectories and scaffolds failed trajectories. Extensive experiments demonstrate that ADMIRE consistently yields over 10% absolute improvement in success rate across different base models on AndroidWorld. Moreover, the method exhibits robust generalizability, achieving strong performance across diverse RL algorithms and heterogeneous environments such as web navigation and embodied tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。