HARP让音频编码更懂音高关系，重建时保留泛音与基频的连贯性。
HARP: Harmonic-Aware Residual Partitioning for Neural Audio Codecs
- 按频率分组逐级细化，每组专注一个频段
- 在语音、音乐上均优于传统方法和并行分解
- 无需改架构，训练时加个损失就行，适合音频生成
基于残差向量量化（RVQ）的神经音频编解码器通常对所有频率一视同仁，导致码本频谱混杂。截断阶段会随机丢弃不同频率成分。并行频带分解虽能解耦频带，但割裂了潜在空间，丢失跨频段连贯性。我们提出HARP（谐波感知残差分区），一种训练策略：将RVQ阶段按频率顺序分组，每组仅优化目标频段，而解码器仍可访问所有低频信息。泛音在基频语境下重建，维持了音高连贯性。HARP无需修改网络结构，仅调整训练损失，推理过程与标准RVQ一致。在语音、音乐及通用音频上，其性能均优于标准RVQ与并行分解方法。MUSHRA听感测试也证实了其主观提升。
原文摘要 · Abstract (English)
Neural audio codecs with residual vector quantization (RVQ) normally treat all frequencies uniformly, so their codebooks become spectrally entangled. Truncating stages then removes an unpredictable mix of frequencies. Parallel band decomposition addresses this by splitting audio into independent bands, but fragments the latent space and loses cross-frequency coherence. We introduce HARP (Harmonic-Aware Residual Partitioning), a training strategy that partitions RVQ stages into frequency-ordered groups where each group refines its target band while the decoder retains access to all lower frequencies. Overtones are reconstructed in the context of their fundamentals, preserving coherence that parallel methods lose. HARP requires no architectural changes; it only modifies the training loss, leaving inference identical to standard RVQ. On speech, music, and general audio, HARP outperforms both standard RVQ and parallel decomposition. MUSHRA listening tests also show perceptual improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。