用焦点调制实现0.55-0.80kbps的流式语音编码，延迟仅80毫秒。
FocalCodec-Stream: Streaming Low-Bitrate Speech Coding via Causal Distillation
- 基于因果蒸馏与轻量重构模块，实现低延迟流式编码
- 0.55-0.80kbps下保持高重建质量与语义信息
- 适合实时语音通信、边缘设备等低带宽场景
神经音频编解码器是现代生成式音频系统的核心组件。尽管近期编解码器在低比特率重建和下游任务表征上表现优异，但多数为非流式，限制了其在实时应用中的使用。本文提出FocalCodec-Stream，一种基于焦点调制的混合编解码器，在0.55至0.80 kbps速率下将语音压缩为单一二进制码本，理论延迟仅为80毫秒。该方法结合WavLM的多阶段因果蒸馏与针对性架构优化，引入轻量级重构模块，在延迟约束下显著提升音质。实验表明，FocalCodec-Stream在相近比特率下优于现有流式编解码器，同时保留语义与声学信息，实现了重建质量、下游任务性能、延迟与效率之间的良好平衡。代码与模型权重将开源于https://github.com/lucadellalib/focalcodec。
原文摘要 · Abstract (English)
Neural audio codecs are a fundamental component of modern generative audio pipelines. Although recent codecs achieve strong low-bitrate reconstruction and provide powerful representations for downstream tasks, most are non-streamable, limiting their use in real-time applications. We present FocalCodec-Stream, a hybrid codec based on focal modulation that compresses speech into a single binary codebook at 0.55 - 0.80 kbps with a theoretical latency of 80 ms. Our approach combines multi-stage causal distillation of WavLM with targeted architectural improvements, including a lightweight refiner module that enhances quality under latency constraints. Experiments show that FocalCodec-Stream outperforms existing streamable codecs at comparable bitrates, while preserving both semantic and acoustic information. The result is a favorable trade-off between reconstruction quality, downstream task performance, latency, and efficiency. Code and checkpoints will be released at https://github.com/lucadellalib/focalcodec.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。