arXiv:2510.07988cs.AI2025-10被引 2

让手机助手在执行任务时主动问用户,更懂真实需求。

ReInAgent: A Context-Aware GUI Agent Enabling Human-in-the-Loop Mobile Task Navigation

  • 用三个智能代理协作,动态管理信息并主动与用户互动。
  • 在复杂任务中成功率比现有方法高25%。
  • 适合需要灵活应变、用户参与度高的手机操作场景。

移动端GUI代理在辅助和自动化手机任务执行方面具有巨大潜力。然而,现有代理大多强调自主运行,忽视了任务执行过程中用户主动参与的必要性。这一缺失削弱了其应对模糊、动态变化和冲突任务场景的适应能力,导致执行结果偏离用户真实需求与偏好。为解决此问题,我们提出ReInAgent——一种上下文感知的多代理框架,通过动态信息管理实现人机协同的移动端任务导航。ReInAgent围绕共享记忆模块集成三个专用代理:信息管理代理负责基于槽位的信息管理并主动与用户交互;决策代理负责冲突感知的规划;反思代理负责任务反思与信息一致性验证。通过持续的上下文信息分析和持续的人机协作,ReInAgent克服了依赖明确静态任务假设的现有方法局限。实验结果表明,ReInAgent能有效化解信息困境,生成更贴近真实用户偏好的执行结果。尤其在涉及信息困境的复杂任务中,其成功率比Mobile-Agent-v2高出25%。

原文摘要 · Abstract (English)

Mobile GUI agents exhibit substantial potential to facilitate and automate the execution of user tasks on mobile phones. However, exist mobile GUI agents predominantly privilege autonomous operation and neglect the necessity of active user engagement during task execution. This omission undermines their adaptability to information dilemmas including ambiguous, dynamically evolving, and conflicting task scenarios, leading to execution outcomes that deviate from genuine user requirements and preferences. To address these shortcomings, we propose ReInAgent, a context-aware multi-agent framework that leverages dynamic information management to enable human-in-the-loop mobile task navigation. ReInAgent integrates three specialized agents around a shared memory module: an information-managing agent for slot-based information management and proactive interaction with the user, a decision-making agent for conflict-aware planning, and a reflecting agent for task reflection and information consistency validation. Through continuous contextual information analysis and sustained user-agent collaboration, ReInAgent overcomes the limitation of existing approaches that rely on clear and static task assumptions. Consequently, it enables more adaptive and reliable mobile task navigation in complex, real-world scenarios. Experimental results demonstrate that ReInAgent effectively resolves information dilemmas and produces outcomes that are more closely aligned with genuine user preferences. Notably, on complex tasks involving information dilemmas, ReInAgent achieves a 25% higher success rate than Mobile-Agent-v2.

人机协同移动代理任务导航

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。