轻量化语音模型在低数据量下仍保持高精度，适合边缘设备部署。
BaldWhisper: Faster Whisper with Head Shearing and Layer Merging
- 通过低秩分解与特征蒸馏压缩嵌入，合并层以减少损失
- 仅用32小时数据训练，性能保留90%，模型缩小48%
- 适合资源受限语言的高效语音识别，尤其适用于移动设备
在数据稀缺场景下对大型预训练变换器进行剪枝极具挑战，通常需要大量重训练数据才能恢复性能。例如，Distill-Whisper 将 Whisper 剪枝40%，并使用21,000小时语音数据重训练，远超多数语言可用数据量。针对仅有32小时语音文本数据的巴姆巴拉语，我们提出一种新型剪枝方法：避免词汇剪枝（因使用者频繁混用语言），改用低秩分解与特征蒸馏压缩嵌入；不删除层，而是合并层以控制性能下降。最终模型保留原模型90%性能，体积缩小48%，在MacBook Air M1上速度提升2.15倍。
原文摘要 · Abstract (English)
Pruning large pre-trained transformers in a data-scarce scenario is challenging, as it often requires massive retraining data to recover performance. For instance, Distill-Whisper prunes Whisper by 40 and retrains on 21,000 hours of speech, far beyond what is available for most languages. Can Whisper be made lighter and faster for edge devices in data-scarce settings? Focusing on Bambara with only 32h of speech-to-text data, we propose a new pruning recipe. Instead of vocabulary pruning, which is unsuitable due to frequent code-switching by Bambara speakers, we compress the embeddings with low-rank decomposition and feature distillation. Rather than removing layers, we merge them to limit performance loss. The final model preserves 90 of the original performance while being 48 smaller and 2.15x faster on a MacBook Air M1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。