用蒙特卡洛模拟在线改进控制策略，显著降低错误率。
On-line Policy Improvement using Monte-Carlo Search
- 通过蒙特卡洛模拟评估各动作长期收益，选择最优动作更新策略。
- 在多种初始策略下，错误率降低至原水平的1/5以下。
- 适合可模拟环境的实时自适应控制场景，如游戏策略优化。
本文提出一种用于自适应控制器在线策略改进的蒙特卡洛仿真算法。在仿真中，利用初始策略在每一步做出决策，统计各可能动作的长期期望回报，选择期望回报最高的动作执行，从而生成改进后的策略。该算法易于并行化，已在IBM SP1和SP2并行RISC超级计算机上实现。实验结果表明，该算法在西洋跳棋领域表现优异，适用于从随机策略到TD-Gammon（一种强大多层神经网络）等多种初始策略。在所有情况下，蒙特卡洛算法均使基础玩家的错误率大幅降低，降幅达五倍或更多。该方法在可模拟环境的其他自适应控制应用中也具有潜力。
原文摘要 · Abstract (English)
We present a Monte-Carlo simulation algorithm for real-time policy improvement of an adaptive controller. In the Monte-Carlo simulation, the long-term expected reward of each possible action is statistically measured, using the initial policy to make decisions in each step of the simulation. The action maximizing the measured expected reward is then taken, resulting in an improved policy. Our algorithm is easily parallelizable and has been implemented on the IBM SP1 and SP2 parallel-RISC supercomputers. We have obtained promising initial results in applying this algorithm to the domain of backgammon. Results are reported for a wide variety of initial policies, ranging from a random policy to TD-Gammon, an extremely strong multi-layer neural network. In each case, the Monte-Carlo algorithm gives a substantial reduction, by as much as a factor of 5 or more, in the error rate of the base players. The algorithm is also potentially useful in many other adaptive control applications in which it is possible to simulate the environment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。