用量子退火优化神经网络,实现高效压缩。
Is Quantum Optimization Ready? An Effort Towards Neural Network Compression using Adiabatic Quantum Computing
- 将模型压缩转化为量子可解的QUBO问题
- 量子退火比经典算法更快且更易找到全局最优
- 适合追求极致压缩效率的硬件部署研究者
量子优化是当前最成熟的量子计算技术,为高效求解复杂组合问题提供了可能。近年来,绝热量子计算(AQC)已被应用于多个领域的关键优化问题。在深度学习中,深度神经网络(DNN)规模不断膨胀以支持更强的预测能力,大规模模型的优化对可持续部署至关重要,但随着模型规模和复杂度的增加,优化难度日益上升。尽管量子优化适用于复杂问题,但其在DNN优化中的应用并不直接,需针对商用量子设备进行彻底重构。本文探索了利用AQC实现卷积神经网络的细粒度剪枝-量化。我们重新设计现有启发式方法,将模型压缩建模为二次无约束二值优化(QUBO)问题,并评估商用量子退火设备提供的解空间。通过系统性的重构尝试,我们证明AQC能够有效压缩实际的DNN模型。实验表明,相较于遗传算法和强化学习等经典算法,AQC在时间效率上更具优势,且更擅长发现全局最优解。
原文摘要 · Abstract (English)
Quantum optimization is the most mature quantum computing technology to date, providing a promising approach towards efficiently solving complex combinatorial problems. Methods such as adiabatic quantum computing (AQC) have been employed in recent years on important optimization problems across various domains. In deep learning, deep neural networks (DNN) have reached immense sizes to support new predictive capabilities. Optimization of large-scale models is critical for sustainable deployment, but becomes increasingly challenging with ever-growing model sizes and complexity. While quantum optimization is suitable for solving complex problems, its application to DNN optimization is not straightforward, requiring thorough reformulation for compatibility with commercially available quantum devices. In this work, we explore the potential of adopting AQC for fine-grained pruning-quantization of convolutional neural networks. We rework established heuristics to formulate model compression as a quadratic unconstrained binary optimization (QUBO) problem, and assess the solution space offered by commercial quantum annealing devices. Through our exploratory efforts of reformulation, we demonstrate that AQC can achieve effective compression of practical DNN models. Experiments demonstrate that adiabatic quantum computing (AQC) not only outperforms classical algorithms like genetic algorithms and reinforcement learning in terms of time efficiency but also excels at identifying global optima.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。