arXiv:2603.17187cs.LG2026-03被引 26

让聊天机器人自我进化,无需停机就能持续学习新技能。

MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wild

  • 用大模型自动提炼失败经验,快速生成新技能
  • 在用户空闲时悄悄优化主模型,准确率提升32%
  • 适合需要长期迭代的智能客服和研究助手

大型语言模型(LLM)代理被广泛用于复杂任务,但部署后的代理通常保持静态,难以随用户需求变化而适应。在包含20多个频道的OpenClaw平台上,现有方法或存储原始轨迹但未提炼知识,或维护固定技能库,或需中断服务重新训练。本文提出MetaClaw,一种持续元学习框架,联合演化基础LLM策略与可复用行为技能库。该框架采用两种互补机制:技能驱动的快速适应通过LLM演进器分析失败轨迹并合成新技能,实现零停机即时改进;机会性策略优化利用云上LoRA微调与基于过程奖励模型的强化学习(RL-PRM),由机会式元学习调度器(OMLS)在用户空闲时段触发,该调度器监控系统空闲状态与日历数据。两者相互增强:优化后的策略生成更优轨迹用于技能合成,丰富技能提供高质量数据用于策略优化。为防止数据污染,采用版本化机制分离支持与查询数据。基于代理架构,MetaClaw可扩展至生产级大模型,无需本地GPU。在MetaClaw-Bench和AutoResearchClaw上的实验表明,技能驱动适应使准确率相对提升最高达32%。完整流水线将Kimi-K2.5准确率从21.4%提升至40.6%,复合鲁棒性提升18.3%。代码已开源。

原文摘要 · Abstract (English)

Large language model (LLM) agents are increasingly used for complex tasks, yet deployed agents often remain static, failing to adapt as user needs evolve. This creates a tension between the need for continuous service and the necessity of updating capabilities to match shifting task distributions. On platforms like OpenClaw, which handle diverse workloads across 20+ channels, existing methods either store raw trajectories without distilling knowledge, maintain static skill libraries, or require disruptive downtime for retraining. We present MetaClaw, a continual meta-learning framework that jointly evolves a base LLM policy and a library of reusable behavioral skills. MetaClaw employs two complementary mechanisms. Skill-driven fast adaptation analyzes failure trajectories via an LLM evolver to synthesize new skills, enabling immediate improvement with zero downtime. Opportunistic policy optimization performs gradient-based updates via cloud LoRA fine-tuning and Reinforcement Learning with a Process Reward Model (RL-PRM). This is triggered during user-inactive windows by the Opportunistic Meta-Learning Scheduler (OMLS), which monitors system inactivity and calendar data. These mechanisms are mutually reinforcing: a refined policy generates better trajectories for skill synthesis, while richer skills provide higher-quality data for policy optimization. To prevent data contamination, a versioning mechanism separates support and query data. Built on a proxy-based architecture, MetaClaw scales to production-size LLMs without local GPUs. Experiments on MetaClaw-Bench and AutoResearchClaw show that skill-driven adaptation improves accuracy by up to 32% relative. The full pipeline advances Kimi-K2.5 accuracy from 21.4% to 40.6% and increases composite robustness by 18.3%. Code is available at https://github.com/aiming-lab/MetaClaw.

元学习自进化智能代理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。