K2-Agent通过分层认知框架提升手机操作复杂任务的完成率。
K^2-Agent: Co-Evolving Know-What and Know-How for Hierarchical Mobile Device Control
- 分离显性知识与操作技能,分层协同进化
- 仅用单次示范即达76.1%成功率,优于基线
- 适合需要长程规划和跨模型泛化的智能体研究
现有移动设备控制智能体在需长程规划与精准操作的复杂任务中表现不佳,常因缺乏任务经验或不熟悉技能执行。本文提出K2-Agent,一种分层框架,通过分离并协同演化陈述性(知道做什么)与程序性(知道怎么做)知识,模拟人类认知。高层推理器基于每项任务的单一示范启动,运行总结-反思-定位-修正(SRLR)循环,通过自进化不断提炼和优化任务级陈述性知识。底层执行器采用课程引导的组相对策略优化(C-GRPO),通过解耦奖励信号构建平衡样本池,并动态注入示范以引导模型自主生成成功轨迹进行训练。在挑战性AndroidWorld基准上,K2-Agent仅使用原始屏幕截图和开源骨干网络,达到76.1%的成功率。此外,该框架展现出强大双泛化能力:其高层陈述性知识可在多种基础模型间迁移,底层程序性技能在ScreenSpot-v2和Android-in-the-Wild(AitW)等未见任务上也表现优异。
原文摘要 · Abstract (English)
Existing mobile device control agents often perform poorly when solving complex tasks requiring long-horizon planning and precise operations, typically due to a lack of relevant task experience or unfamiliarity with skill execution. We propose K2-Agent, a hierarchical framework that models human-like cognition by separating and co-evolving declarative (knowing what) and procedural (knowing how) knowledge for planning and execution. K2-Agent's high level reasoner is bootstrapped from a single demonstration per task and runs a Summarize-Reflect-Locate-Revise (SRLR) loop to distill and iteratively refine task-level declarative knowledge through self-evolution. The low-level executor is trained with our curriculum-guided Group Relative Policy Optimization (C-GRPO), which (i) constructs a balanced sample pool using decoupled reward signals and (ii) employs dynamic demonstration injection to guide the model in autonomously generating successful trajectories for training. On the challenging AndroidWorld benchmark, K2-Agent achieves a 76.1% success rate using only raw screenshots and open-source backbones. Furthermore, K2-Agent shows powerful dual generalization: its high-level declarative knowledge transfers across diverse base models, while its low-level procedural skills achieve competitive performance on unseen tasks in ScreenSpot-v2 and Android-in-the-Wild (AitW).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。