针对语音大模型设计自适应量化方法，压缩模型体积并提速，性能损失极小。
StableQuant: Layer Adaptive Post-Training Quantization for Speech Foundation Models
- 按层动态调整量化范围，结合尺度分布与整体表现分析。
- 8比特量化后模型大小减至四分之一，推理速度翻倍，错误率提升不足0.3%。
- 适用于各类语音基础模型，尤其适合无微调场景的部署优化。
本文提出StableQuant，一种面向广泛使用的语音基础模型（SFMs）的新型自适应后训练量化（PTQ）算法。尽管PTQ在压缩大语言模型时表现良好，但直接应用于依赖独特特征提取结构的语音模型时效果不佳。StableQuant通过分析各层的尺度分布与整体性能，自适应确定每层的量化范围，实现对不同网络结构的最优量化效果。我们在两个语音基础模型HuBERT和wav2vec2.0上评估该方法，在自动语音识别（ASR）任务中表现优于传统PTQ方法。使用8比特量化时，模型尺寸减少至原体积的四分之一，推理速度提升一倍，词错误率（WER）下降低于0.3%。
原文摘要 · Abstract (English)
In this paper, we propose StableQuant, a novel adaptive post-training quantization (PTQ) algorithm for widely used speech foundation models (SFMs). While PTQ has been successfully employed for compressing large language models (LLMs) due to its ability to bypass additional fine-tuning, directly applying these techniques to SFMs may not yield optimal results, as SFMs utilize distinct network architecture for feature extraction. StableQuant demonstrates optimal quantization performance regardless of the network architecture type, as it adaptively determines the quantization range for each layer by analyzing both the scale distributions and overall performance. We evaluate our algorithm on two SFMs, HuBERT and wav2vec2.0, for an automatic speech recognition (ASR) task, and achieve superior performance compared to traditional PTQ methods. StableQuant successfully reduces the sizes of SFM models to a quarter and doubles the inference speed while limiting the word error rate (WER) performance drop to less than 0.3% with 8-bit quantization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。