ChunkFormer让长音频语音识别在低显存GPU上高效运行
ChunkFormer: Masked Chunking Conformer For Long-Form Speech Transcription
- 采用分块处理+相对右文机制，突破长音频处理瓶颈
- 80GB GPU可处理16小时音频，错误率降低7.7%以上
- 无填充批处理设计，大幅降低显存与计算成本
工业级语音识别部署面临硬件资源管理挑战，尤其在长达数小时的长音频转录任务中。大型Conformer模型在80GB GPU上仅能处理15分钟音频，且输入长度不一导致标准批处理产生大量冗余填充，加剧资源消耗与延迟。为此，本文提出ChunkFormer，一种基于分块处理与相对右文机制的高效语音识别模型，可在80GB GPU上处理长达16小时的音频，较当前最优模型FastConformer提升1.5倍处理时长。该模型在长音频任务上实现最高7.7%的词错误率绝对降低，同时保持短音频任务的精度。通过引入无填充批处理技术，相较于传统批处理，执行时间与内存占用减少3倍以上，显著降低各类语音识别系统尤其是实际应用中的GPU资源开销。
原文摘要 · Abstract (English)
Deploying ASR models at an industrial scale poses significant challenges in hardware resource management, especially for long-form transcription tasks where audio may last for hours. Large Conformer models, despite their capabilities, are limited to processing only 15 minutes of audio on an 80GB GPU. Furthermore, variable input lengths worsen inefficiencies, as standard batching leads to excessive padding, increasing resource consumption and execution time. To address this, we introduce ChunkFormer, an efficient ASR model that uses chunk-wise processing with relative right context, enabling long audio transcriptions on low-memory GPUs. ChunkFormer handles up to 16 hours of audio on an 80GB GPU, 1.5x longer than the current state-of-the-art FastConformer, while also boosting long-form transcription performance with up to 7.7% absolute reduction on word error rate and maintaining accuracy on shorter tasks compared to Conformer. By eliminating the need for padding in standard batching, ChunkFormer's masked batching technique reduces execution time and memory usage by more than 3x in batch processing, substantially reducing costs for a wide range of ASR systems, particularly regarding GPU resources for models serving in real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。