量化让Whisper模型更快更小，适合在手机等设备上部署。
Quantization for OpenAI's Whisper Models: A Comparative Analysis
- 用INT4/INT5/INT8三种方法量化模型，减少计算开销。
- 量化后延迟降19%，模型大小减45%，准确率基本不变。
- 适合想在低算力设备运行ASR的人参考。
自动语音识别（ASR）模型广泛应用于字幕生成、语音翻译和实时转录。本文研究了Whisper及其两个变体：一个针对实时语音流优化，另一个用于离线转录。值得注意的是，这些模型会产生幻觉内容，降低转录可靠性。此外，更大模型的延迟更高，在资源受限设备上部署困难。本研究分析了三个Whisper模型的异同，定性比较其能力差异。随后，量化评估了模型量化对延迟的影响，并验证其在边缘设备部署的可行性。基于开源LibriSpeech数据集，论文评估了whispercpp在三种量化方法（INT4、INT5、INT8）下的词错误率（WER）与延迟表现。结果表明，量化使延迟降低19%，模型体积减少45%，同时保持转录准确率。研究为不同Whisper模型的适用场景及边缘部署提供了参考。所有代码、数据集和实现细节均开源至GitHub：https://github.com/allisonandreyev/WhisperQuantization.git
原文摘要 · Abstract (English)
Automated speech recognition (ASR) models have gained prominence for applications such as captioning, speech translation, and live transcription. This paper studies Whisper and two model variants: one optimized for live speech streaming and another for offline transcription. Notably, these models have been found to generate hallucinated content, reducing transcription reliability. Furthermore, larger model variants exhibit increased latency and pose challenges for deployment on resource-constrained devices. This study analyzes the similarities and differences between three Whisper models, qualitatively examining their distinct capabilities. Next, this study quantifies the impact of model quantization on latency and evaluates its viability for edge deployment. Using the open source LibriSpeech dataset, this paper evaluates the word error rate (WER) along with latency analysis of whispercpp using 3 quantization methods (INT4, INT5, INT8). Results show that quantization reduces latency by 19\% and model size by 45\%, while preserving transcription accuracy. These findings provide insights into the optimal use cases of different Whisper models and edge device deployment possibilities. All code, datasets, and implementation details are available in a public GitHub repository: https://github.com/allisonandreyev/WhisperQuantization.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。