用相同随机数减少模拟规划方差，提升决策准确性
Using Common Random Numbers for Simulation-based Planning with Rollouts
- 采用共用随机数降低仿真中动作效用估计的方差
- 在深层回溯时显著提升任务表现，合成任务验证有效
- 适合需要高精度决策的金融与游戏类应用
基于模拟的规划通过反复采样生成轨迹并估计动作效用，在随机环境中广泛应用。本文研究在模拟过程中使用共用随机数的影响，提出一种简单方法，在回溯深度超过一定阈值时可严格降低相对效用估计的方差。合成任务实验表明该方案显著提升任务性能。其实际价值体现在两个场景：一是养老金发放任务中的单步前瞻规划，二是将著名的UCT算法应用于棋类游戏Ludo的部署。
原文摘要 · Abstract (English)
Simulation-based planning with rollouts is a widely-deployed technique for decision making in stochastic environments. The primary instrument of simulation-based planning is a sampling model, which is repeatedly called to generate trajectories and estimate the utilities of available actions. Among the actions thus explored, one with the maximum estimated utility is then executed. In this paper, we examine the effect of using common random numbers in the simulation process. We obtain a simple recipe for (provably) reducing variance in relative utility when simulations invoke a rollout policy beyond some depth. Experiments on synthetic tasks confirm that our scheme improves task performance. The broader significance of our innovation is apparent from two practical applications: (1) single-step lookahead planning in a pension-disbursement task, and (2) a deployment of the well-known UCT algorithm for the game of Ludo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。