基于频谱信息的低秩微调提升语音验证精度
Spectral-Aware Low-Rank Adaptation for Speaker Verification
- 仅在主频谱空间内进行参数微调，避免噪声干扰
- 在VoxCeleb1和CN-Celeb1上显著提升验证准确率
- 适合对表征能力要求高的语音验证任务
已有研究显示，预训练模型权重矩阵的主奇异向量包含关键知识，而小奇异值对应的向量可能包含噪声或不可靠信息。因此，不约束频谱空间的LoRA类参数高效微调方法，在需要高表征能力的任务中可能效果不佳。本文通过将预训练权重矩阵的频谱信息融入微调过程，提升现有参数高效微调技术性能。重点研究了对前若干个奇异向量的加性调整策略，通过奇异值分解（SVD）定位主要频谱空间，并限制微调在此空间内进行。在VoxCeleb1和CN-Celeb1上的大量语音验证实验表明，该方法显著提升了微调性能。代码已公开于https://github.com/lizhepolyu/SpectralFT。
原文摘要 · Abstract (English)
Previous research has shown that the principal singular vectors of a pre-trained model's weight matrices capture critical knowledge. In contrast, those associated with small singular values may contain noise or less reliable information. As a result, the LoRA-based parameter-efficient fine-tuning (PEFT) approach, which does not constrain the use of the spectral space, may not be effective for tasks that demand high representation capacity. In this study, we enhance existing PEFT techniques by incorporating the spectral information of pre-trained weight matrices into the fine-tuning process. We investigate spectral adaptation strategies with a particular focus on the additive adjustment of top singular vectors. This is accomplished by applying singular value decomposition (SVD) to the pre-trained weight matrices and restricting the fine-tuning within the top spectral space. Extensive speaker verification experiments on VoxCeleb1 and CN-Celeb1 demonstrate enhanced tuning performance with the proposed approach. Code is released at https://github.com/lizhepolyu/SpectralFT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。