在设备端实现低延迟高精度语音识别，性能超越云端主流系统。
WhisperKit: On-device Real-time ASR with Billion-Scale Transformers
- 通过模型压缩与推理优化，在设备端实现实时语音识别。
- 延迟仅0.46秒，词错误率2.2%，优于云端主流系统。
- 适合对隐私和实时性要求高的场景，如医疗记录、会议转录。
实时自动语音识别（ASR）是许多机器学习商业应用的基础，包括实时字幕、语音输入、会议转录和医疗记录。准确率与延迟是企业选型的核心指标。本文提出WhisperKit，一个面向设备端的实时语音识别优化推理系统，显著优于主流云端方案。我们对比了多种服务器端系统，包括前沿模型OpenAI gpt-4o-transcribe、专有模型Deepgram nova-3以及开源模型Fireworks large-v3-turbo。结果表明，WhisperKit在0.46秒延迟下达到最低的2.2%词错误率（WER），实现最佳性能平衡。文中详细阐述了系统的各项优化策略。
原文摘要 · Abstract (English)
Real-time Automatic Speech Recognition (ASR) is a fundamental building block for many commercial applications of ML, including live captioning, dictation, meeting transcriptions, and medical scribes. Accuracy and latency are the most important factors when companies select a system to deploy. We present WhisperKit, an optimized on-device inference system for real-time ASR that significantly outperforms leading cloud-based systems. We benchmark against server-side systems that deploy a diverse set of models, including a frontier model (OpenAI gpt-4o-transcribe), a proprietary model (Deepgram nova-3), and an open-source model (Fireworks large-v3-turbo).Our results show that WhisperKit matches the lowest latency at 0.46s while achieving the highest accuracy 2.2% WER. The optimizations behind the WhisperKit system are described in detail in this paper.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。