让大模型智能体用树状结构管理经验,提升复杂任务的推理能力。
Tree-of-Experience: Hierarchical Experience Management for Self-Evolving Agents

- 构建树状经验结构,对齐大模型的分步推理过程。
- 在24点游戏上准确率提升31.4%,在金融任务中效率提高41.24%。
- 适合需要持续进化和跨任务迁移的智能体系统研究者。
持续自我进化要求大语言模型智能体将环境交互转化为可靠且可复用的经验。现有方法通常优化单个轨迹或从相关轨迹中抽象共性知识,但其经验表示常与底层推理过程脱节,限制了反馈归因、跨任务迁移以及更新和检索效率,尤其在仅提供结果级反馈的复杂推理任务中。为此,我们提出树状经验(Tree-of-Experience, ToE)框架,将经验组织为共享的分析视角与推理路径树,其可靠性通过环境结果校准,支持系统化更新、迁移与高效检索。在 extsc{Game of 24}和 extsc{FinEvolveBench}上的实验表明,ToE显著提升问题解决性能与效率:在 extsc{Game of 24}上相比无经验的思维链基线,准确率相对提升31.4%;在 extsc{FinEvolveBench}上,跨12个评估场景,平均提升tsIC达41.24%,而传统经验管理方法反而常低于无经验基线。
原文摘要 · Abstract (English)
Continual self-evolution requires LLM agents to transform environmental interactions into reliable and reusable experience. Existing methods typically refine individual trajectories or abstract shared knowledge from related trajectories, but their experience representations are often disconnected from the underlying reasoning process. This limits feedback attribution, cross-task transfer, and update and retrieval efficiency, particularly in complex reasoning tasks with outcome-level feedback. To overcome this limitation, we propose \textbf{T}ree-\textbf{o}f-\textbf{E}xperience (ToE), a structured experience-management framework that aligns experience organization with the hierarchical reasoning process of LLM agents. Specifically, ToE organizes the experience into a shared tree of analytical perspectives and reasoning paths, whose reliability is calibrated through environmental outcomes to support systematic updating, transfer, and efficient retrieval. The experimental results on \textsc{Game of 24} and \textsc{FinEvolveBench} show that ToE substantially improves both problem-solving performance and efficiency. On \textsc{Game of 24}, ToE achieves a 31.4\% relative improvement in accuracy over the experience-free ToT baseline. On \textsc{FinEvolveBench}, ToE improves tsIC by an average of 41.24\% over the experience-free pipeline across 12 evaluation settings, whereas conventional experience-management methods often underperform experience-free baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。