让手机智能体像人一样好奇,主动查资料补知识,提升复杂任务成功率。
Curiosity Driven Knowledge Retrieval for Mobile Agents
- 用好奇心分数判断何时需要查外部资料,触发信息检索。
- 检索内容生成结构化AppCard,包含功能、参数、界面等关键信息,平均提效6个百分点。
- 特别适合多步骤和跨应用任务,适合研究移动端智能体的开发者参考。
移动智能体在智能手机自动化方面已取得进展,但在复杂应用中仍受限于知识不全和对未见环境的泛化能力弱。本文提出一种基于好奇心的知识检索框架,将执行过程中的不确定性形式化为好奇心分数。当该分数超过阈值时,系统会从文档、代码仓库和历史轨迹中检索外部信息。检索内容被组织成结构化的AppCard,编码功能语义、参数规范、界面映射和交互模式。执行过程中,增强型智能体有选择地将相关AppCard融入推理流程,从而弥补知识盲区,提升规划可靠性。在AndroidWorld基准上的评估显示,该方法在多种主干模型上均实现稳定提升,结合GPT-5时达到88.8%的新状态领先成功率,平均提升6个百分点。分析表明,AppCard对多步及跨应用任务尤为有效,性能提升依赖于主干模型。案例研究进一步验证了其降低歧义、缩短探索时间、支持稳定执行轨迹的能力。任务轨迹公开可访问:https://lisalsj.github.io/Droidrun-appcard/。
原文摘要 · Abstract (English)
Mobile agents have made progress toward reliable smartphone automation, yet performance in complex applications remains limited by incomplete knowledge and weak generalization to unseen environments. We introduce a curiosity driven knowledge retrieval framework that formalizes uncertainty during execution as a curiosity score. When this score exceeds a threshold, the system retrieves external information from documentation, code repositories, and historical trajectories. Retrieved content is organized into structured AppCards, which encode functional semantics, parameter conventions, interface mappings, and interaction patterns. During execution, an enhanced agent selectively integrates relevant AppCards into its reasoning process, thereby compensating for knowledge blind spots and improving planning reliability. Evaluation on the AndroidWorld benchmark shows consistent improvements across backbones, with an average gain of six percentage points and a new state of the art success rate of 88.8\% when combined with GPT-5. Analysis indicates that AppCards are particularly effective for multi step and cross application tasks, while improvements depend on the backbone model. Case studies further confirm that AppCards reduce ambiguity, shorten exploration, and support stable execution trajectories. Task trajectories are publicly available at https://lisalsj.github.io/Droidrun-appcard/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。