提出一体化混合精度量化方法，显著提升语音大模型压缩效率与速度。
Effective and Efficient Mixed Precision Quantization of Speech Foundation Models
- 将精度学习与参数量化整合为单一压缩阶段，避免分步处理开销。
- 最高实现8.6倍无损压缩比，系统压缩时间提速1.9倍。
- 适用于需高效部署语音大模型的场景，如移动端或边缘设备。
本文提出一种新型混合精度量化方法，将混合精度学习与量化参数估计深度融合于单一模型压缩阶段。在LibriSpeech数据集上对微调后的wav2vec2.0-base和HuBERT-large模型进行实验，结果表明，所提方法相较分别执行精度学习与参数量化的两阶段基线，可使无损压缩比分别提升1.7倍和1.9倍；同时未引入统计学意义上的词错误率（WER）增加，保持与32位全精度模型相当的性能。wav2vec2.0-base和HuBERT-large模型的系统压缩时间相比两阶段基线分别缩短1.9倍和1.5倍，且生成的模型具有更低的WER。其中表现最佳的3.5比特混合精度量化HuBERT-large模型，相较32位全精度系统实现8.6倍的无损压缩比。
原文摘要 · Abstract (English)
This paper presents a novel mixed-precision quantization approach for speech foundation models that tightly integrates mixed-precision learning and quantized model parameter estimation into one single model compression stage. Experiments conducted on LibriSpeech dataset with fine-tuned wav2vec2.0-base and HuBERT-large models suggest the resulting mixed-precision quantized models increased the lossless compression ratio by factors up to 1.7x and 1.9x over the respective uniform-precision and two-stage mixed-precision quantized baselines that perform precision learning and model parameters quantization in separate and disjointed stages, while incurring no statistically word error rate (WER) increase over the 32-bit full-precision models. The system compression time of wav2vec2.0-base and HuBERT-large models is reduced by up to 1.9 and 1.5 times over the two-stage mixed-precision baselines, while both produce lower WERs. The best-performing 3.5-bit mixed-precision quantized HuBERT-large model produces a lossless compression ratio of 8.6x over the 32-bit full-precision system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。