arXiv:2511.10395cs.LGcs.AI2025-11被引 62

用自问自答机制让智能体自动学习,省去人工数据和低效试错。

AgentEvolver: Towards Efficient Self-Evolving Agent System

  • 让智能体自己提问生成任务,减少人工数据依赖。
  • 通过重用经验与混合策略,探索效率提升,样本利用率更高。
  • 适合想低成本训练自进化智能体的研究者或开发者。

由大语言模型驱动的自主智能体有望显著提升人类生产力,在多样化环境中实现推理、调用工具并执行复杂任务。然而,当前方法开发成本高且效率低,通常依赖手工构建的任务数据集以及需大量随机探索的强化学习(RL)流程,导致数据构建成本过高、探索效率低下、样本利用不足。为解决这些问题,我们提出AgentEvolver——一个自演化智能体系统,利用大语言模型的语义理解与推理能力驱动智能体自主学习。该系统引入三种协同机制:(i) 自我提问,实现新颖环境中的好奇心驱动任务生成,降低对手工数据集的依赖;(ii) 自我导航,通过经验复用与混合策略引导提升探索效率;(iii) 自我归因,基于状态与动作贡献度分配差异化奖励,增强样本效率。将这些机制整合至统一框架中,使智能体具备可扩展、低成本、持续进化的特性。初步实验表明,AgentEvolver在探索效率、样本利用率及适应速度上均优于传统基于RL的基线方法。

原文摘要 · Abstract (English)

Autonomous agents powered by large language models (LLMs) have the potential to significantly enhance human productivity by reasoning, using tools, and executing complex tasks in diverse environments. However, current approaches to developing such agents remain costly and inefficient, as they typically require manually constructed task datasets and reinforcement learning (RL) pipelines with extensive random exploration. These limitations lead to prohibitively high data-construction costs, low exploration efficiency, and poor sample utilization. To address these challenges, we present AgentEvolver, a self-evolving agent system that leverages the semantic understanding and reasoning capabilities of LLMs to drive autonomous agent learning. AgentEvolver introduces three synergistic mechanisms: (i) self-questioning, which enables curiosity-driven task generation in novel environments, reducing dependence on handcrafted datasets; (ii) self-navigating, which improves exploration efficiency through experience reuse and hybrid policy guidance; and (iii) self-attributing, which enhances sample efficiency by assigning differentiated rewards to trajectory states and actions based on their contribution. By integrating these mechanisms into a unified framework, AgentEvolver enables scalable, cost-effective, and continual improvement of agent capabilities. Preliminary experiments indicate that AgentEvolver achieves more efficient exploration, better sample utilization, and faster adaptation compared to traditional RL-based baselines.

智能体自进化强化学习大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。