arXiv:2601.22528cs.AI2026-01被引 3

无需训练的自进化记忆系统,让GUI代理更智能地适应复杂操作。

Darwinian Memory: A Training-Free Self-Regulating Memory System for GUI Agent Evolution

  • 将任务分解为可复用单元,通过生存竞争筛选最优路径。
  • 成功率达18.0%提升,执行稳定性提高33.9%,延迟降低。
  • 适合需要长期跨应用操作的自动化场景,无需额外训练成本。

多模态大语言模型(MLLM)代理虽能实现图形用户界面(GUI)自动化,但在长时程、跨应用任务中受限于有限的上下文窗口。现有记忆系统难以适应动态GUI环境,存在高层意图与低层执行粒度不匹配、静态经验积累导致上下文污染的问题,引发幻觉。为此,我们提出达尔文记忆系统(DMS),一种自演化架构,将记忆构建为受“适者生存”法则驱动的动态生态系统。DMS将复杂轨迹分解为独立可复用单元,实现组合灵活性,并通过效用驱动的自然选择机制追踪路径生存价值,主动剔除低效路径并抑制高风险计划。这种演化压力促使代理生成更优策略。在真实世界多应用基准上的大量实验表明,DMS可在无训练成本和架构开销的前提下,显著提升通用MLLM的表现,平均成功率提升18.0%,执行稳定性提高33.9%,同时降低任务延迟,验证了其作为GUI任务高效自演化记忆系统的有效性。

原文摘要 · Abstract (English)

Multimodal Large Language Model (MLLM) agents facilitate Graphical User Interface (GUI) automation but struggle with long-horizon, cross-application tasks due to limited context windows. While memory systems provide a viable solution, existing paradigms struggle to adapt to dynamic GUI environments, suffering from a granularity mismatch between high-level intent and low-level execution, and context pollution where the static accumulation of outdated experiences drives agents into hallucination. To address these bottlenecks, we propose the Darwinian Memory System (DMS), a self-evolving architecture that constructs memory as a dynamic ecosystem governed by the law of survival of the fittest. DMS decomposes complex trajectories into independent, reusable units for compositional flexibility, and implements Utility-driven Natural Selection to track survival value, actively pruning suboptimal paths and inhibiting high-risk plans. This evolutionary pressure compels the agent to derive superior strategies. Extensive experiments on real-world multi-app benchmarks validate that DMS boosts general-purpose MLLMs without training costs or architectural overhead, achieving average gains of 18.0% in success rate and 33.9% in execution stability, while reducing task latency, establishing it as an effective self-evolving memory system for GUI tasks.

GUI自动化自进化记忆系统MLLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。