arXiv:2412.08845quant-phcs.AI2024-12被引 24

用量子计算加速多智能体强化学习,提升分布式场景下的训练速度与可扩展性。

Quantum-Train-Based Distributed Multi-Agent Reinforcement Learning

  • 基于量子电路生成神经网络参数,实现参数量对数级压缩。
  • 在分布式环境中并行运行多个量子处理单元,收敛速度更快。
  • 融合量子与经典计算,适合高维复杂任务的实时应用。

本文提出量子训练驱动的分布式多智能体强化学习(Dist-QTRL),通过引入量子计算原理解决传统强化学习的可扩展性问题。量子训练强化学习(QTRL)利用参数化量子电路高效生成神经网络参数,在保持模型性能的同时,将可训练参数维度降至 $poly(/log(N))$ 级别,并借助量子纠缠实现更优的数据表征。该框架适用于分布式多智能体环境,多个智能体以量子处理单元(QPUs)形式并行运作,显著加快收敛速度并增强系统可扩展性。此外,该框架可通过分布式量子训练降低经典神经网络参数量,再由经典CPU/GPU完成推理,形成混合量子-HPC架构,进一步优化实际应用效率。本文给出了Dist-QTRL的数学建模及收敛性分析,并通过实验证明其优于集中式QTRL模型,尤其在高维分布式场景中实现显著提速,且不牺牲准确性。本研究为实际应用中的可扩展量子增强强化学习系统奠定了基础。

原文摘要 · Abstract (English)

In this paper, we introduce Quantum-Train-Based Distributed Multi-Agent Reinforcement Learning (Dist-QTRL), a novel approach to addressing the scalability challenges of traditional Reinforcement Learning (RL) by integrating quantum computing principles. Quantum-Train Reinforcement Learning (QTRL) leverages parameterized quantum circuits to efficiently generate neural network parameters, achieving a \(poly(\log(N))\) reduction in the dimensionality of trainable parameters while harnessing quantum entanglement for superior data representation. The framework is designed for distributed multi-agent environments, where multiple agents, modeled as Quantum Processing Units (QPUs), operate in parallel, enabling faster convergence and enhanced scalability. Additionally, the Dist-QTRL framework can be extended to high-performance computing (HPC) environments by utilizing distributed quantum training for parameter reduction in classical neural networks, followed by inference using classical CPUs or GPUs. This hybrid quantum-HPC approach allows for further optimization in real-world applications. In this paper, we provide a mathematical formulation of the Dist-QTRL framework and explore its convergence properties, supported by empirical results demonstrating performance improvements over centric QTRL models. The results highlight the potential of quantum-enhanced RL in tackling complex, high-dimensional tasks, particularly in distributed computing settings, where our framework achieves significant speedups through parallelization without compromising model accuracy. This work paves the way for scalable, quantum-enhanced RL systems in practical applications, leveraging both quantum and classical computational resources.

强化学习量子计算分布式多智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。