为机器优化音频编码，实现超低码率下性能几乎无损。
Task-Specific Audio Coding for Machines: Machine-Learned Latent Features Are Codes for That Machine
- 用任务特定损失+残差向量量化压缩下游模型中间特征。
- 码率低于200 bps，语音识别与分类任务性能损失极小。
- 适合需要轻量化部署的语音任务，如边缘设备应用。
神经音频编解码器借助量化算法显著提升了各类语音/音频任务表现。尽管人类感知追求高保真重建，但面向机器的音频编码（ACoM）更关注高效压缩与下游任务性能，忽略感知细节。本文提出一种高效的ACoM方法，可对已训练语音/音频下游模型的任意中间特征表示进行压缩与量化。该方法结合任务特定损失与残差向量量化（RVQ）损失，在保持下游模型性能的前提下实现超低码率（低于200 bps）。生成的分词器可适配不同码率与模型规模，便于灵活部署。在自动语音识别与音频分类任务上验证，该方法展现出优异性能及广泛的任务与架构适用潜力，通过适当正则化进一步提升效果。
原文摘要 · Abstract (English)
Neural audio codecs, leveraging quantization algorithms, have significantly impacted various speech/audio tasks. While high-fidelity reconstruction is paramount for human perception, audio coding for machines (ACoM) prioritizes efficient compression and downstream task performance, disregarding perceptual nuances. This work introduces an efficient ACoM method that can compress and quantize any chosen intermediate feature representation of an already trained speech/audio downstream model. Our approach employs task-specific loss guidance alongside residual vector quantization (RVQ) losses, providing ultra-low bitrates (i.e., less than 200 bps) with a minimal loss of the downstream model performance. The resulting tokenizer is adaptable to various bitrates and model sizes for flexible deployment. Evaluated on automatic speech recognition and audio classification, our method demonstrates its efficacy and potential for broader task and architectural applicability through appropriate regularization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。