arXiv:2605.29324cs.CLcs.CV2026-05被引 1

让手机界面智能体学会主动记关键信息,解决长期任务记忆难题。

STAMP: Training Explicit Memory for Mobile GUI Agents in Controllable and Scalable Virtual Environments

论文配图:STAMP: Training Explicit Memory for Mobile GUI Agents in Controllable and Scalable Virtual Environments
图 1 · 摘自论文原文
  • 通过可控虚拟环境,让智能体学习何时记、记什么、何时用。
  • 在新基准上超越现有模型,记忆准确率与任务韧性显著提升。
  • 适合研究长期决策、移动代理与可解释记忆的学者和开发者。

移动端图形界面智能体在即时反应控制上表现良好,但在需要记忆的长周期任务中常失败,根源在于有限上下文窗口与高密度截图之间的矛盾。为节省上下文,智能体必须逐步丢弃旧视觉历史,永久丢失关键临时信息。现有以动作为中心的数据集无法教会智能体何时何地显式记忆,且扩充真实世界数据成本高昂且缺乏交互验证。为此,我们提出STAMP框架,通过可控虚拟环境训练智能体的显式记忆:在合成任务中程序化注入确定性记忆变量,精确控制需记忆的内容、编码时机与检索时机,从而大规模生成可验证的监督数据,并支持环境驱动的在线强化学习。在新提出的Memory-World基准上,Stamp-GUI智能体达到当前最优性能,刷新基准记录,展现出卓越的记忆准确性与任务鲁棒性,同时保持强大的通用移动端导航能力。

原文摘要 · Abstract (English)

Mobile GUI agents excel at immediate reactive control but frequently fail in realistic, long-horizon tasks that require memory. This failure stems from a fundamental conflict between limited context windows and token-heavy screenshots. To save the limited context, agents must progressively discard older visual history, permanently losing crucial transient information. Furthermore, existing action-centric datasets fail to teach agents what or when to explicitly memorize, and augmenting static real-world data is prohibitively expensive and lacks interactive verification. To resolve this, we present STAMP, a framework that trains explicit memory in mobile agents through controllable virtual environments, where deterministic memory variables are programmatically injected into synthesized tasks to control what must be memorized, when it should be encoded, and when it must later be retrieved, thereby producing verifiable supervised data at scale and enabling online reinforcement learning through environment-driven reward feedback. Evaluated on our newly introduced Memory-World benchmark, the resulting Stamp-GUI agent achieves state-of-the-art performance among GUI-specialized models and sets a new high watermark on our Memory-World benchmark, demonstrating exceptional memory accuracy and task resilience while maintaining strong general mobile navigation capabilities.

移动智能体显式记忆虚拟环境长周期任务

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。