用动态电路重用量子比特,让量子强化学习在小设备上也能处理长序列任务。
Scalable Quantum Reinforcement Learning on NISQ Devices with Dynamic-Circuit Qubit Reuse and Grover Optimization
- 用中途测量与重置技术复用固定数量量子比特,避免随步骤增加而扩展硬件
- 在7个量子比特上实现10步决策过程,相比传统方法减少66%量子比特占用
- 适合在噪声中等的量子设备上运行,为大规模量子强化学习提供可扩展方案
本文提出一种可扩展且资源高效的量子强化学习框架,突破了多步量子马尔可夫决策过程(QMDP)中量子比特线性增长的瓶颈。该框架融合量子马尔可夫决策过程建模、动态电路执行与格罗弗幅值放大,构建统一的原生量子架构。环境动态完全编码于量子希尔伯特空间,实现状态-动作序列的相干叠加与无需中间经典转换的直接量子交互。核心创新在于采用中段测量与重置的动态执行模型,将固定物理量子寄存器跨多轮交互循环使用,使轨迹保真度与静态展开方案一致,同时将物理量子比特需求从7×T降低至恒定7个,实现从O(T)到O(1)的复杂度跃迁。轨迹回报通过量子算术评估,高回报轨迹经幅值放大提升采样概率。仿真验证轨迹保真度保持不变,量子比特使用量相较静态设计降低66%。在IBM Heron类处理器上的实验表明该方法在含噪中等规模量子硬件上可行,为大规模原生量子强化学习奠定了可扩展、资源高效的基础。
原文摘要 · Abstract (English)
A scalable and resource-efficient quantum reinforcement learning framework is presented that eliminates the linear qubit-scaling barrier in multi-step quantum Markov decision processes (QMDPs). The proposed framework integrates a QMDP formulation, dynamic-circuit execution, and Grover-based amplitude amplification into a unified quantum-native architecture. Environment dynamics are encoded entirely within quantum Hilbert space, enabling coherent superposition over state-action sequences and a direct quantum agent-environment interface without intermediate quantum-to-classical conversion. The central contribution is a dynamic execution model for multi-step QMDPs that employs mid-circuit measurement and reset to recycle a fixed physical quantum register across sequential interactions. This approach preserves trajectory fidelity relative to a static unrolled QMDP, generating identical state-action sequences while reducing the physical qubit requirement from 7xT to a constant 7, independent of the interaction horizon T. Thus, the qubit complexity of multi-step QMDPs is transformed from O(T) to O(1) while maintaining functional equivalence at the level of trajectory generation. Trajectory returns are evaluated via quantum arithmetic, and high-return trajectories are marked and amplified using amplitude amplification to increase their sampling probability. Simulations confirm preservation of trajectory fidelity with a 66% qubit reduction compared to a static design. Experimental execution on an IBM Heron-class processor demonstrates feasibility on noisy intermediate-scale quantum hardware, establishing a scalable and resource-efficient foundation for large-scale quantum-native reinforcement learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。