让大模型智能体学会纠错回滚，避免错误累积。
Generator-Assistant Stepwise Rollback Framework for Large Language Model Agent
- 用生成器与助手协同，助手发现错动作就触发回滚。
- 在三个基准测试中显著优于现有方法。
- 可无缝接入其他系统，适合需要可靠推理的场景。
大型语言模型（LLM）智能体通常采用逐步推理框架，将思考与行动交替进行以完成任务。然而，这一范式存在深层的单次执行问题：每个生成的中间思维都会被直接纳入决策轨迹，无论其正确与否，导致不可逆的错误传播。为解决该问题，本文提出一种新型框架——生成器-助手逐步回滚（GA-Rollback），以提升LLM智能体的决策能力。GA-Rollback利用生成器与环境交互，同时由助手审查生成器的每一步动作，一旦检测到错误动作即触发回滚操作。此外，我们还设计了两种针对回滚场景的优化策略，进一步增强其有效性。大量实验表明，GA-Rollback在三个广泛使用的基准测试上显著优于多个强基线方法。分析进一步揭示，GA-Rollback可作为稳健的即插即用模块，与其它方法无缝集成。
原文摘要 · Abstract (English)
Large language model (LLM) agents typically adopt a step-by-step reasoning framework, in which they interleave the processes of thinking and acting to accomplish the given task. However, this paradigm faces a deep-rooted one-pass issue whereby each generated intermediate thought is plugged into the trajectory regardless of its correctness, which can cause irreversible error propagation. To address the issue, this paper proposes a novel framework called Generator-Assistant Stepwise Rollback (GA-Rollback) to induce better decision-making for LLM agents. Particularly, GA-Rollback utilizes a generator to interact with the environment and an assistant to examine each action produced by the generator, where the assistant triggers a rollback operation upon detection of incorrect actions. Moreover, we introduce two additional strategies tailored for the rollback scenario to further improve its effectiveness. Extensive experiments show that GA-Rollback achieves significant improvements over several strong baselines on three widely used benchmarks. Our analysis further reveals that GA-Rollback can function as a robust plug-and-play module, integrating seamlessly with other methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。