通过信息价值动态调整搜索，让智能体更高效地做决策。
Leveraging the Value of Information in POMDP Planning
- 根据信息价值决定是否处理观测数据，避免无效分支。
- 在多个基准测试中优于传统方法，尤其在时间受限时表现更好。
- 适合需要快速决策的机器人、自动驾驶等实时系统。
部分可观测马尔可夫决策过程（POMDP）为状态与转移不确定下的规划提供了严谨框架。尽管在求解大规模POMDP方面取得进展，但在有限规划时间内获得高性能策略仍面临挑战，主要源于维度灾难与历史灾难。许多POMDP问题中，信息价值（VOI）——即通过推理观测带来的期望性能提升——在信念空间中变化显著。本文提出一种动态规划框架，通过基于每个信念下的信息价值有条件地处理观测，利用该结构。在此基础上，我们提出信息价值蒙特卡洛规划（VOIMCP），一种蒙特卡洛树搜索算法，通过在信息价值较低时选择性忽略观测信息，减少不必要的观测分支，从而更高效分配计算资源。我们为该信息价值推理框架提供近似最优性理论保证，并推导出VOIMCP的非渐近收敛界。仿真评估表明，VOIMCP在多个POMDP基准上优于基线方法。
原文摘要 · Abstract (English)
Partially observable Markov decision processes (POMDPs) offer a principled formalism for planning under state and transition uncertainty. Despite advances made towards solving large POMDPs, obtaining performant policies under limited planning time remains a major challenge due to the curse of dimensionality and the curse of history. For many POMDP problems, the value of information (VOI) - the expected performance gain from reasoning about observations - varies over the belief space. We introduce a dynamic programming framework that exploits this structure by conditionally processing observations based on the value of information at each belief. Building on this framework, we propose Value of Information Monte Carlo planning (VOIMCP), a Monte Carlo Tree Search algorithm that allocates computational effort more efficiently by selectively disregarding observation information when the VOI is low, avoiding unnecessary branching of observations. We provide theoretical guarantees on the near-optimality of our VOI reasoning framework and derive non-asymptotic convergence bounds for VOIMCP. Simulation evaluations demonstrate that VOIMCP outperforms baselines on several POMDP benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。