根据策略质量动态调整搜索深度，提升强化学习效率。
Adaptive Mixing of Policies from Searching and Policies from Learning
- 通过神经网络与蒙特卡洛树搜索混合策略，自适应调节搜索强度。
- 在三个符号任务上超越AlphaZero、DQN和ADP的性能表现。
- 适合追求高效强化学习的算法研究者与工程实践者。
背景：通过搜索/规划生成训练目标的蒸馏方法在强化学习中已被证明有效，但搜索耗时过长。目标：不再固定每步进行相同深度的搜索（通常按固定步数执行），而是根据策略网络先验的质量动态减少搜索深度。方法：提出Flexer架构，每一步混合神经网络策略与蒙特卡洛树搜索（MCTS）策略；当网络策略的模仿误差增大或环境模型方差上升时，混合因子更倾向于MCTS策略。结果：在三个玩具符号问题上的实验中，Flexer优于固定深度搜索的AlphaZero版本，以及DQN和ADP方法。
原文摘要 · Abstract (English)
Background: Distillation of training targets generated thru search/planning has proven useful in reinforcement learning, but search can take exceedingly long. Objectives: Rather than perform search to the same depth every time (typically at a fixed period of steps), reduce the search depth proportionally to the quality of the policy network priors. Methods: We describe Flexer, an architecture that, for each step, mixes the policy from a neural network and the policy from Monte Carlo tree search. The mixing factor favors the MCTS policy as the policy imitation error of the network and the environment models' variance increases. Results: Flexer outperforms a version of AlphaZero (and DQN and ADP) for some experiments on three toy symbolic problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。