将语音识别模型压缩到1.6GB，实现在低核数CPU上实时识别。
VibeVoice-ASR-BitNet Technical Report
- 针对不同模块采用混合量化：VAE用INT8+SIMD优化，语言模型用二值权重。
- 在1.6~2.3倍提速下，仅轻微损失精度，实测RTF<1。
- 专为边缘设备设计，适合部署在ARM/x86低功耗设备上。
我们提出VibeVoice-ASR-BitNet，一个面向边缘CPU实时推理的压缩版VibeVoice-ASR模型。针对各阶段计算特性，采用异构量化策略：VAE声学分词器使用全链路INT8量化（I8_S）并结合内核融合与SIMD优化；自回归语言模型则采用BitNet风格的三值权重（I2_S）。为在高压缩下保持精度，采用渐进式量化感知训练。推理时，基于ggml框架实现定制化SIMD内核与融合算子，支持ARM和x86平台，可在低线程数CPU上实现实时识别（RTF < 1）。该模型大小约1.6 GB，相比Whisper.cpp快1.6–2.3倍，且与FP16基线相比仅出现轻微准确率下降。
原文摘要 · Abstract (English)
We present VibeVoice-ASR-BitNet, a compressed variant of VibeVoice-ASR optimized for real-time inference on edge CPUs. We apply heterogeneous quantization tailored to the computational characteristics of each stage: the VAE acoustic tokenizer uses full-pipeline INT8 quantization (I8_S) with kernel fusion and SIMD optimization, while the autoregressive language model adopts BitNet-style ternary weights (I2_S). To preserve accuracy under aggressive compression, we employ a progressive quantization-aware training strategy. For inference, we implement custom SIMD kernels and fused operators within the ggml framework targeting both ARM and x86 platforms, achieving real-time recognition (RTF < 1) on low-thread-count CPUs. VibeVoice-ASR-BitNet is 1.6--2.3x faster than Whisper.cpp at comparable model sizes (~1.6 GB), with only modest accuracy degradation compared to the FP16 baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。