用轻量微调技术提升方言识别效率，内存降73%、速度提2.1倍。
Dialect Identification Using Resource-Efficient Fine-Tuning Approaches
- 将语言模型的高效内存微调方法移植到语音模型，适配方言识别任务。
- 在KeSpeech数据集上对Whisper模型微调，内存减少73.25%，训练提速2.1倍。
- 适合资源受限场景下的语音方言识别应用，兼顾性能与效率。
方言识别（DI）是从语音信号中识别同一语言不同方言的任务，有助于提升下游语音任务表现，尤其在说话人方言特征强烈时。然而，微调语音模型进行DI计算成本高，内存占用大。近期研究采用参数高效微调（PEFT）方法，虽节省参数但未显著改善内存和训练速度。本文探索了原本用于自然语言处理的内存高效微调（MEFT）方法，并应用于通用预训练语音模型。通过对比多种MEFT方法的GPU内存使用与训练速度，以Whisper模型在KeSpeech数据集上识别六种普通话子方言为例，实现最高73.25%的内存降低和2.1倍的训练加速，同时保持与标准微调及PEFT相当的识别准确率。
原文摘要 · Abstract (English)
Dialect Identification (DI) is a task to recognize different dialects within the same language from a speech signal. DI can help to improve the downstream speech related tasks even when speakers have a strong dialect. However, fine-tuning a speech model for tasks like DI is expensive in terms of computation cost and memory requirement. Recent studies have explored fine-tuning pre-trained speech models for tasks like DI using Parameter-Efficient Fine-Tuning (PEFT) methods, which offer parameter efficiency but limited improvement in memory efficiency and training speed. To address these challenges, we explore Memory-Efficient Fine-Tuning (MEFT) methods, originally proposed for language processing, and apply them to the general-purpose pre-trained speech model. We then comprehensively analyze the GPU memory usage and fine-tuning speed based on various MEFT methods. As a case study, we fine-tune the Whisper model to identify six Mandarin subdialects from the KeSpeech dataset, reducing GPU memory usage by up to 73.25% and accelerating training speed by a factor of 2.1, while maintaining accuracy comparable to vanilla fine-tuning and PEFT methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。