arXiv:2606.04815cs.LGcs.AI2026-06被引 2

让智能体像人一样边做边学,实时优化技能和推理能力。

Learning While Acting: A Skill-Enhanced Test-Time Co-Evolution Framework for Online Lifelong Learning Agents

论文配图:Learning While Acting: A Skill-Enhanced Test-Time Co-Evolution Framework for Online Lifelong Learning Agents
图 1 · 摘自论文原文
  • 用验证器反馈引导技能生成,确保学到的技能真正有用。
  • 测试时持续将行为轨迹转为奖励信号,提升模型性能7个百分点。
  • 适合需要长期适应新任务的动态环境智能体研究者。

长期学习对在动态交互环境中运行的大语言模型智能体至关重要。然而,现有长期学习智能体在长周期任务中通常依赖静态参数下的离散技能或过往经验检索,无法像人类学习者一样在测试时持续吸收反馈。为此,我们提出技能增强型测试时协同演化框架(LifeSkill),一种面向在线长期学习智能体的两阶段强化学习方法。具体而言,我们设计了验证器引导的技能学习机制,通过多个技能条件策略滚动的平均验证成功率来奖励候选技能,促使模型生成真正有助于解决问题的技能,而非仅文本上合理的技能。此外,我们引入在线技能内化机制,通过将技能条件轨迹转化为奖励信号,在测试时持续优化策略模型。这使智能体能直接将推理能力内化到参数中,避免经验检索导致的上下文膨胀。在LifelongAgentBench上的实验表明,LifeSkill相比现有基线平均性能提升7个百分点。

原文摘要 · Abstract (English)

Lifelong learning is essential for Large Language Model (LLM) agents operating in dynamic, interactive environments. However, existing lifelong learning agents for long-horizon tasks typically depend on discrete skill or past experiences retrieval with static parameters during inference, which prevents them from continuously internalizing test-time feedback like human learners. To bridge this gap, we propose Skill-enhanced Test-Time Co-Evolution (\texttt{LifeSkill}), a two-stage reinforcement learning framework for Online Lifelong Learning Agents. Specifically, we design Verifier-Guided Skill Learning that addresses the lack of direct supervision for skill extraction by rewarding candidate skills according to the average verifier success of multiple skill-conditioned policy rollouts, encouraging the model to generate skills that are useful for solving tasks rather than merely plausible in text. Furthermore, we introduce Online Skill Internalization, which continuously improves the policy model during test-time interaction by transforming skill-conditioned trajectories into reward signals. This enables the agent to directly internalize reasoning capabilities into its parameters, avoiding the context bloat of experience retrieval. Experiments on LifelongAgentBench show that LifeSkill improves average performance by 7 absolute points by comparing with existing lifelong agent baselines.

长期学习强化学习智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。