改进信息价值评估，让智能体在延迟决策中更懂何时该收集信息。
Increasing the Value of Information During Planning in Uncertain Environments
- 在POMCP算法中引入熵来优化信息收集动作的选择
- 在走廊问题上显著优于传统POMCP算法
- 适合需延迟决策的现实场景，如机器人导航、远程监控
先前研究表明，许多现实问题可通过在线算法快速且近乎最优地求解部分可观马尔可夫决策过程（POMDP）。然而，在信息获取与使用存在显著时间延迟的问题中，现有方法未能充分考虑信息的价值。这导致关键的信息收集动作被忽略，从而造成次优决策。本研究提出一种新算法，通过在POMCP的UCB1启发式中加入熵项，增强对信息收集动作价值的评估。实验在走廊问题（hallway problem）上进行，结果表明新算法显著优于标准POMCP。
原文摘要 · Abstract (English)
Prior studies have demonstrated that for many real-world problems, POMDPs can be solved through online algorithms both quickly and with near optimality. However, on an important set of problems where there is a large time delay between when the agent can gather information and when it needs to use that information, these solutions fail to adequately consider the value of information. As a result, information gathering actions, even when they are critical in the optimal policy, will be ignored by existing solutions, leading to sub-optimal decisions by the agent. In this research, we develop a novel solution that rectifies this problem by introducing a new algorithm that improves upon state-of-the-art online planning by better reflecting on the value of actions that gather information. We do this by adding Entropy to the UCB1 heuristic in the POMCP algorithm. We test this solution on the hallway problem. Results indicate that our new algorithm performs significantly better than POMCP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。