无需数值优化，高效求解部分可观测决策问题
Partially Observable Reference Policy Programming: Solving POMDPs Sans Numerical Optimisation
- 通过深度采样未来历史并渐进更新策略求解POMDP
- 性能损失由平均采样误差决定，优于传统最大误差约束
- 适合动态环境下的实时决策，如直升机应急场景
本文提出一种名为部分可观测参考策略编程（Partially Observable Reference Policy Programming）的新型在线近似POMDP求解算法。该算法在不依赖数值优化的前提下，能够深度采样有意义的未来历史轨迹，并同步实现策略的渐进式更新。我们为该算法的核心机制提供了理论保证：性能损失被限制在采样近似误差的平均值范围内，而非传统方法中的最大值，这一特性在在线规划中采样稀疏的情况下尤为重要。在两个大规模动态环境问题上的实验验证了理论结果，包括一个需约150次规划步的科西嘉地区直升机紧急场景，结果表明该求解器显著优于现有在线基准方法。
原文摘要 · Abstract (English)
This paper proposes Partially Observable Reference Policy Programming, a novel anytime online approximate POMDP solver which samples meaningful future histories very deeply while simultaneously forcing a gradual policy update. We provide theoretical guarantees for the algorithm's underlying scheme which say that the performance loss is bounded by the average of the sampling approximation errors rather than the usual maximum, a crucial requirement given the sampling sparsity of online planning. Empirical evaluations on two large-scale problems with dynamically evolving environments -- including a helicopter emergency scenario in the Corsica region requiring approximately 150 planning steps -- corroborate the theoretical results and indicate that our solver considerably outperforms current online benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。