arXiv:2607.17038cs.AI2026-07

用强化学习+自我纠错提升智能体长程决策能力

Reward-Driven LLM Agent Workflows: Synthesizing POMDP Routing and Self-Correction for Autonomous Decision-Making

  • 基于部分可观测马尔可夫决策过程与自校正奖励模型,动态优化决策路径
  • 在ALFWorld和WebShop上任务成功率提升24.5%,轨迹效率显著改善
  • 适合研究复杂多步自主系统或想减少幻觉的LLM应用开发者

本文针对当前大语言模型智能体在长周期规划、稀疏奖励分配及动态环境交互中的关键技术挑战,设计并优化了一种智能体工作流。该架构融合视觉、语言、生成、图结构、多模态、强化学习与智能体等多种人工智能范式。不同于依赖静态提示且缺乏鲁棒感知-动作回路的传统基线模型,本方法引入基于部分可观测马尔可夫决策过程(POMDP)的路由机制,并结合内部自校正奖励模型,在执行前评估决策轨迹。通过整合多模态输入与先进强化学习技术(如近端策略优化和价值函数逼近),智能体维持长期结构化记忆,动态调整推理路径以缓解误差累积。在ALFWorld具身仿真环境与WebShop在线导航基准上的实验证明,相比主流基线(如标准ReAct框架),任务成功率绝对提升24.5%,轨迹效率显著提高。全面消融实验确认了奖励驱动批判模块在抑制幻觉率方面的关键作用。该研究将强化学习与图结构记忆的理论基础与自主智能体工作流相衔接,最终提供了一个可实践、可扩展的复杂多步自主系统人工智能开发参考框架。代码已公开于 https://github.com/01Amez/RLAW_Implementation。

原文摘要 · Abstract (English)

This paper addresses key technical challenges in current large language model (LLM) agent applications, including long-horizon planning, sparse reward attribution, and dynamic environmental interaction, by designing and optimizing an intelligent agent workflow. The proposed architecture is based on the synthesis of core AI paradigms: Visual, Language, Generative, Graph, Multimodal, Reinforcement, and Agent Intelligence. Unlike conventional baseline models that rely on static prompting and lack robust perception-action loops, our approach introduces a Partially Observable Markov Decision Process (POMDP) routing mechanism. This mechanism is augmented with an internal, self-correcting reward model that evaluates decision trajectories before execution. By integrating multimodal inputs and advanced reinforcement learning principles (such as proximal policy optimization and value function approximation), the agent maintains long-term structural memory and dynamically adapts its reasoning pathways to mitigate error accumulation. Empirical experiments on the ALFWorld embodied simulation environment and the WebShop online navigation benchmark demonstrate a 24.5% absolute improvement in task success rate and trajectory efficiency over mainstream baselines like the standard ReAct framework. Comprehensive ablation studies confirm the significant contribution of the reward-driven critique module in suppressing hallucination rates. This research bridges theoretical foundations of reinforcement learning and graph-based memory with autonomous agent workflows. Ultimately, the resulting architecture offers a practical, scalable reference framework for developing artificial intelligence technologies in complex, multi-step autonomous systems. Code is available at https://github.com/01Amez/RLAW_Implementation.

强化学习智能体决策优化自校正

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。