让Whisper在手机NPU上实时运行，通过消除冗余计算大幅降低延迟和功耗。
NPUsper: Eliminating Redundant Computation for Real-Time Whisper on Mobile NPUs
- 基于解码器注意力模式在线检测幻觉词，减少输入填充与状态传递。
- 采用可控展开策略，将自回归解码拆分为块图执行，减少缓存计算与调度开销。
- 适合移动端语音转写场景，尤其对低延迟、低功耗有要求的设备开发者。
我们提出NPUsper，一个面向移动NPU的实时语音转录系统，通过消除冗余计算提升Whisper的效率。为避免先前流式系统中高开销的填充操作，NPUsper基于解码器交叉注意力中的时序模式，实时检测幻觉词，使每轮推理仅处理短音频片段并最小化状态传递。针对移动NPU高效执行，我们提出受控展开策略，将自回归解码以K步块图为形式执行，移除不必要的键值缓存计算，降低图调度开销。相比基线系统，NPUsper实现最高4.84倍更低的每词延迟、最高33.2倍更低的首次词时间（TTFT），以及最高88.64%的平均功耗降低，同时保持相当的转录准确率。代码已开源：https://github.com/npusper/NPUsper。
原文摘要 · Abstract (English)
We present NPUsper, a live transcription system that makes Whisper efficient on mobile NPUs by eliminating redundant computation. To avoid the heavy padding used by prior streaming systems, NPUsper detects hallucinated tokens online from temporal patterns in decoder cross-attention, allowing each inference round to process short audio inputs with minimal carryover. For efficient mobile-NPU execution, we propose controlled unrolling, which executes autoregressive decoding as K-step chunk graphs, removing unnecessary KV-cache computation and reducing graph-dispatch overhead. NPUsper achieves up to 4.84x lower per-word latency, up to 33.2x lower time-to-first-token (TTFT), and up to 88.64% lower average power consumption compared with baselines, while maintaining comparable transcription accuracy. The code is available at https://github.com/npusper/NPUsper.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。