让音频丢失隐藏模型在接收时自动优化,用收到的片段自我训练提升修复效果。
Self-Supervised Test-Time Tuning for Packet Loss Concealment
- 用接收到的音频片段自动生成训练信号,动态调整模型参数。
- 在非因果和因果场景下均显著提升音频修复质量,最高提升2.1 dB PESQ。
- 无需参考音频或修改模型结构,适合实时通信和录音修复场景。
音频丢包隐藏(PLC)用于重建接收端缺失的音频包,通常使用部署后参数固定的训练模型。这将PLC模型视为静态,尽管每通电话或录音都通过已接收包传递了信号特异性信息。本文提出TTT-PLC,一种仅利用接收包的自监督测试时调优框架,可动态适配现有PLC模型。该方法通过合成掩蔽可用信号部分生成监督信号,训练模型以原生PLC目标掩盖这些部分,再用优化后的模型重建真实丢包。无需干净参考信号、外部适应数据或架构修改。我们在两种部署场景中评估TTT-PLC:非因果场景下可对完整文件重复自监督调优,提供每文件调优上限;因果场景下音频流式传输,仅对已完成的过去块进行调优,更新参数仅影响未来音频。在两个公开的PLC主干模型上验证:FRN(循环全带语音PLC模型)与PARCnet(网络音乐的混合自回归-神经模型)。结果表明,预训练的PLC系统无需在推理时保持固定,仍可观察到的信号部分能有效作为训练信号,提升同一信号的隐藏性能。
原文摘要 · Abstract (English)
Packet loss concealment (PLC) reconstructs audio packets that are missing at the receiver, usually with a trained model whose parameters remain fixed at deployment time. This treats the PLC model as static, even though each call or recording exposes signal-specific information through the packets that did arrive. We present TTT-PLC, a self-supervised test-time tuning framework that adapts existing PLC models using only those received packets. The method creates supervision by synthetically masking portions of the available signal, training the model to conceal them with its native PLC objective, and then using the adapted model to reconstruct the true packet losses. No clean reference signal, external adaptation data, or architectural modification is required. We study TTT-PLC in two deployment settings. In the non-causal setting, the received file is available before reconstruction, allowing repeated self-supervised adaptation passes and providing a per-file adaptation ceiling. In the causal setting, audio is streamed without revising emitted samples; adaptation is performed only on completed past blocks, and updated parameters affect only future audio. We instantiate the framework on two public PLC backbones, FRN, a recurrent full-band speech PLC model, and PARCnet, a hybrid autoregressive-neural model for networked music. Across these settings, the results show that pretrained PLC systems do not need to be treated as fixed at inference time, the still-observed portions of a lossy signal can provide an effective training signal for improving concealment on that same signal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。