用低秩近似压缩语音识别模型，提速降耗还更准
LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximation
- 通过主成分分析提取激活值低秩特征，用低秩矩阵替代原线性变换
- 压缩Whisper Large-v3编码器超50%，体积接近Medium版但更准确
- 适合部署在资源受限设备上的高精度语音识别应用
现代自动语音识别（ASR）模型如OpenAI的Whisper采用深度编码器-解码器结构，其中编码器因计算密集成为高效部署的瓶颈。本文提出LiteASR，一种针对ASR编码器的低秩压缩方法，显著降低推理开销的同时保持转录准确性。该方法利用中间激活值的强低秩特性：通过小规模校准数据集进行主成分分析（PCA），将线性变换替换为一系列低秩矩阵乘法，并进一步优化自注意力机制以在低维空间运行。评估结果显示，该方法可将Whisper large-v3编码器大小压缩超过50%，达到Whisper medium级别体积，且转录准确率更高，从而在准确率与效率之间建立了新的帕累托前沿。LiteASR代码已开源至https://github.com/efeslab/LiteASR。
原文摘要 · Abstract (English)
Modern automatic speech recognition (ASR) models, such as OpenAI's Whisper, rely on deep encoder-decoder architectures, and their encoders are a critical bottleneck for efficient deployment due to high computational intensity. We introduce LiteASR, a low-rank compression scheme for ASR encoders that significantly reduces inference costs while maintaining transcription accuracy. Our approach leverages the strong low-rank properties observed in intermediate activations: by applying principal component analysis (PCA) with a small calibration dataset, we approximate linear transformations with a chain of low-rank matrix multiplications, and further optimize self-attention to work in reduced dimensionality. Evaluation results show that our method can compress Whisper large-v3's encoder size by over 50%, matching Whisper medium's size with better transcription accuracy, thereby establishing a new Pareto frontier of accuracy and efficiency. The code of LiteASR is available at https://github.com/efeslab/LiteASR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。