让手机界面智能体学会自我进化记忆，提升长期任务表现
UI-Mem: Self-Evolving Experience Memory for Online Reinforcement Learning in Mobile GUI Agents
- 构建分层经验记忆，存储流程、技能和失败模式
- 通过分组采样保持策略多样性，实现跨任务迁移
- 自进化循环持续更新记忆，适配智能体能力变化
在线强化学习为通过直接环境交互提升GUI智能体提供了有前景的范式。然而，其效果严重受限于长周期任务中的信用分配效率低下以及因缺乏经验迁移导致的任务重复错误。为此，我们提出UI-Mem，一种基于分层经验记忆的新型框架，增强在线GUI强化学习。与传统回放缓冲区不同，该记忆积累结构化知识，包括高层工作流、子任务技能和失败模式，并以参数化模板形式存储，支持跨任务与跨应用迁移。为有效整合记忆指导，我们引入分层分组采样机制，在每轮轨迹组内注入不同层次的引导，维持结果多样性，推动无指导策略内化指导行为。此外，自进化循环持续抽象新策略与错误，使记忆始终与智能体演化策略对齐。在在线GUI基准上的实验表明，UI-Mem显著优于传统强化学习基线和静态重用策略，并展现出对未见应用的强大泛化能力。
原文摘要 · Abstract (English)
Online Reinforcement Learning (RL) offers a promising paradigm for enhancing GUI agents through direct environment interaction. However, its effectiveness is severely hindered by inefficient credit assignment in long-horizon tasks and repetitive errors across tasks due to the lack of experience transfer. To address these challenges, we propose UI-Mem, a novel framework that enhances GUI online RL with a Hierarchical Experience Memory. Unlike traditional replay buffers, our memory accumulates structured knowledge, including high-level workflows, subtask skills, and failure patterns. These experiences are stored as parameterized templates that enable cross-task and cross-application transfer. To effectively integrate memory guidance into online RL, we introduce Stratified Group Sampling, which injects varying levels of guidance across trajectories within each rollout group to maintain outcome diversity, driving the unguided policy toward internalizing guided behaviors. Furthermore, a Self-Evolving Loop continuously abstracts novel strategies and errors to keep the memory aligned with the agent's evolving policy. Experiments on online GUI benchmarks demonstrate that UI-Mem significantly outperforms traditional RL baselines and static reuse strategies, with strong generalization to unseen applications. Project page: https://ui-mem.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。