用量子神经网络突破传统微调的表达瓶颈,让大模型更轻更快地适应新任务。
Quantum-Enhanced LLM Efficient Fine Tuning
- 将量子神经网络与张量网络结合,在低秩空间中实现混合参数微调。
- 参数减少76%的情况下,训练损失降低17%,测试性能提升17%。
- 为未来量子增强型通用人工智能提供可工程化的技术基础。
低秩适配(LoRA)通过低秩矩阵近似实现了预训练语言模型的高效微调,在多种场景中表现良好。然而,在复杂任务或高秩依赖设置下,其表达能力受限,可能影响模型适应性。为克服经典低秩近似在微调大语言模型时的表达瓶颈,本文提出量子张量混合适配(QTHA),一种将量子神经网络(QNN)与张量网络结合的参数高效微调方法。QTHA通过将预训练权重分解为量子神经网络与张量网络表示,在低秩空间中探索量子张量混合微调,利用量子态叠加突破经典秩限制。实验表明,QTHA在参数高效微调中性能可媲美或超越LoRA:相比LoRA,QTHA减少76%可训练参数,训练损失最多降低17%,测试集性能最多提升17%。该研究不仅实现了量子资源对十亿级参数模型的轻量级适配,还验证了基于大模型任务驱动的量子硬件优化可行性,建立了首个面向未来量子增强型通用人工智能(AGI)系统的工程化基础。
原文摘要 · Abstract (English)
Low-Rank Adaptation (LoRA) enables efficient fine-tuning of pre-trained language models through low-rank matrix approximation, achieving effectiveness in many scenarios. However, its representation capacity is constrained in complex tasks or high-rank dependency settings, potentially limiting model adaptability. To overcome the expressive bottleneck in classical low-rank approximation for fine-tuning large language models (LLMs), we propose Quantum Tensor Hybrid Adaptation (QTHA), a parameter-efficient fine-tuning method that integrates a quantum neural network (QNN) with a tensor network. QTHA explores quantum tensor hybrid fine-tuning within low-rank spaces by decomposing pre-trained weights into quantum neural network and tensor network representations, leveraging quantum state superposition to overcome classical rank limitations. Experiments demonstrate that QTHA achieves performance comparable to or surpassing LoRA in parameter-efficient fine-tuning. Compared to LoRA, QTHA reduces trainable parameters by 76% while reducing training loss by up to 17% and improving test set performance by up to 17% within the same training steps. This research not only enables lightweight adaptation of quantum resources to the billion-parameter models but also validates the feasibility of quantum hardware optimization driven by LLM tasks. It establishes the first engineering-ready foundation for future quantum-enhanced Artificial General Intelligence (AGI) systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。