让智能体在自然语言环境中自建状态空间,实现可解释决策
State-Centric Decision Process

- 智能体逐步构建自然语言谓词作为状态,行动后验证并固化为真实状态
- 在5个基准上实现无训练最优,长时序任务优势更明显
- 支持状态级归因、故障定位等分析,适合需要可解释性的场景
语言环境如网页浏览器、代码终端和交互式模拟器仅输出原始文本,缺乏马尔可夫决策过程(MDP)分析所需的运行时结构:无显式状态空间、无观测到状态的映射、无认证转移、无终止条件。本文提出状态中心决策过程(SDP),一种运行时框架,让智能体在行动过程中逐谓词构建这些缺失要素。每一步中,智能体承诺一个描述世界应有样貌的自然语言谓词,执行动作使其成立,并用观测验证。通过验证的谓词成为认证状态,生成的轨迹包含任务诱导的状态空间、观测到状态的映射、认证转移和终止条件。我们在涵盖规划、科学探索、网络推理和多跳问答的五个基准上评估了SDP,所有任务均取得无训练最优结果,且随着决策时域增长,优势愈发显著。认证轨迹还支持反应式智能体无法实现的分析,包括谓词级信用分配、故障定位、部分进展度量和模块化算子替换。
原文摘要 · Abstract (English)
Language environments such as web browsers, code terminals, and interactive simulations emit raw text rather than states, and provide none of the runtime structure that MDP analysis requires. No explicit state space, no observation-to-state mapping, no certified transitions, and no termination criterion. We introduce the State-Centric Decision Process (SDP), a runtime framework that constructs these missing inputs by having the agent build them, predicate by predicate, as it acts. At each step the agent commits to a natural-language predicate describing how the world should look, takes an action to make it true, and checks the observation against it. Predicates that pass become certified states, and the resulting trajectory carries the four objects language environments do not provide, namely a task-induced state space, an observation-to-state mapping, certified transitions, and a termination criterion. We evaluate SDP on five benchmarks spanning planning, scientific exploration, web reasoning, and multi-hop question answering. SDP achieves the best training-free results on all five, with the advantage widening as the horizon grows. The certified trajectories additionally support analyses unavailable to reactive agents, including per-predicate credit assignment, failure localization, partial-progress measurement, and modular operator replacement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。