用量子参数化实现超高效微调，参数量随维度增长近乎不变。
Quantum-PEFT: Ultra parameter-efficient fine-tuning
- 采用保罗矩阵参数化实现全秩但极省参的量子单元。
- 参数量随维度增长仅对数级增加，远低于LoRA的线性增长。
- 在语言与视觉任务中显著提升参数效率，适合资源受限场景。
本文提出Quantum-PEFT，利用量子计算实现参数高效的微调（PEFT）。不同于其他添加型PEFT方法（如低秩适配LoRA），Quantum-PEFT基于一个全秩但出人意料地参数高效的量子酉参数化。通过保罗矩阵参数化，可训练参数数量随环境维度增长仅呈对数关系，而传统LoRA方法为线性增长。随着维度增大，Quantum-PEFT的可训练参数量远低于最低秩的LoRA，提升了参数效率，同时保持了竞争力的性能。我们在语言和视觉多个迁移学习基准上应用Quantum-PEFT，验证了其在参数效率上的显著优势。
原文摘要 · Abstract (English)
This paper introduces Quantum-PEFT that leverages quantum computations for parameter-efficient fine-tuning (PEFT). Unlike other additive PEFT methods, such as low-rank adaptation (LoRA), Quantum-PEFT exploits an underlying full-rank yet surprisingly parameter efficient quantum unitary parameterization. With the use of Pauli parameterization, the number of trainable parameters grows only logarithmically with the ambient dimension, as opposed to linearly as in LoRA-based PEFT methods. Quantum-PEFT achieves vanishingly smaller number of trainable parameters than the lowest-rank LoRA as dimensions grow, enhancing parameter efficiency while maintaining a competitive performance. We apply Quantum-PEFT to several transfer learning benchmarks in language and vision, demonstrating significant advantages in parameter efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。