量子经典混合模型成功玩转雅达利游戏,表现媲美传统模型。
A quantum-classical reinforcement learning model to play Atari games
- 用量子电路+经典编码层构建混合强化学习模型
- 在Pong上成功通关,在Breakout上得分接近经典模型
- 揭示量子与经典组件协同的关键超参数设计
近期强化学习进展表明,基于参数化量子电路的量子学习模型可作为深度学习的替代方案。尽管全量子模型在特定人工环境中展现指数级加速潜力,而近场量子电路也已能解决OpenAI Gym基准任务,但其能否应对高维观测空间的复杂问题仍未知。本文提出一种混合模型,结合参数化量子电路与经典特征编码和后处理层,用于应对雅达利游戏。构建一个受相同架构限制的经典模型作为参照。数值实验表明,该混合模型可在Pong环境中成功通关,并在Breakout任务中达到与经典模型相当的分数。研究还揭示了影响量子-经典组件协同的关键超参数设置。本工作深化了对近场量子学习模型的理解,为其实现真实强化学习场景部署迈出重要一步。
原文摘要 · Abstract (English)
Recent advances in reinforcement learning have demonstrated the potential of quantum learning models based on parametrized quantum circuits as an alternative to deep learning models. On the one hand, these findings have shown the ultimate exponential speed-ups in learning that full-blown quantum models can offer in certain -- artificially constructed -- environments. On the other hand, they have demonstrated the ability of experimentally accessible PQCs to solve OpenAI Gym benchmarking tasks. However, it remains an open question whether these near-term QRL techniques can be successfully applied to more complex problems exhibiting high-dimensional observation spaces. In this work, we bridge this gap and present a hybrid model combining a PQC with classical feature encoding and post-processing layers that is capable of tackling Atari games. A classical model, subjected to architectural restrictions similar to those present in the hybrid model is constructed to serve as a reference. Our numerical investigation demonstrates that the proposed hybrid model is capable of solving the Pong environment and achieving scores comparable to the classical reference in Breakout. Furthermore, our findings shed light on important hyperparameter settings and design choices that impact the interplay of the quantum and classical components. This work contributes to the understanding of near-term quantum learning models and makes an important step towards their deployment in real-world RL scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。