arXiv:2508.02046cs.ROcs.LG2025-08ACL被引 13

首个统一处理GUI与实体导航的智能体,让机器学会跨场景操作。

NaviMaster: Learning a Unified Policy for GUI and Embodied Navigation Tasks

  • 用统一轨迹生成方法同时构建GUI和实体导航数据
  • 在混合数据上训练,提升跨任务泛化能力
  • 设计距离感知奖励,加速高效学习

GUI导航与实体导航虽各自进展迅速,但长期孤立发展,数据集与训练方式各异。本文发现二者均可建模为马尔可夫决策过程(MDP),提出首个统一框架NaviMaster,实现两类任务的融合。该方法 (i) 设计视觉目标轨迹采集管道,以同一形式生成两类任务轨迹;(ii) 在混合数据上采用统一强化学习框架,增强泛化性能;(iii) 提出新颖的距离感知奖励机制,提升学习效率。在跨领域基准测试中,NaviMaster在GUI导航、空间可操作性预测及实体导航任务上均优于现有最优模型。消融实验验证了统一训练策略、数据混合方法与奖励设计的有效性。代码、数据与模型检查点已公开于https://iron-boyy.github.io/navimaster-page/。

原文摘要 · Abstract (English)

Recent advances in Graphical User Interface (GUI) and embodied navigation have driven progress, yet these domains have largely evolved in isolation, with disparate datasets and training paradigms. In this paper, we observe that both tasks can be formulated as Markov Decision Processes (MDP), suggesting a foundational principle for their unification. Hence, we present NaviMaster, the first unified agent capable of unifying GUI navigation and embodied navigation within a single framework. Specifically, NaviMaster (i) proposes a visual-target trajectory collection pipeline that generates trajectories for both GUI and embodied tasks using a single formulation. (ii) employs a unified reinforcement learning framework on the mix data to improve generalization. (iii) designs a novel distance-aware reward to ensure efficient learning from the trajectories. Through extensive experiments on out-of-domain benchmarks, NaviMaster is shown to outperform state-of-the-art agents in GUI navigation, spatial affordance prediction, and embodied navigation. Ablation studies further demonstrate the efficacy of our unified training strategy, data mixing strategy, and reward design. Our codes, data, and checkpoints are available at https://iron-boyy.github.io/navimaster-page/.

导航统一框架强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。