用ASR自验证让语音合成几乎零崩溃，且推理时无需额外开销。
Reliable Neural-Codec Text-to-Speech by ASR Self-Verification and Distillation: Near-Zero Catastrophic Failures Across Models and Codecs
- 通过ASR回环验证，多候选生成筛选可消除大部分语音合成崩溃。
- N=2时在标准数据集上失败率趋近于零，N=4在困难语句上同样有效。
- 蒸馏后单次生成即可保持鲁棒性，适合部署在实时语音系统中。
开放的自回归神经编码器语音合成（TTS）模型在常规输入下表现优秀，但在部分语句上会出现静音、提前终止或重复/幻觉内容等随机性灾难性失败。我们发现这类失败极易修复。采用单一格式鲁棒指标（通过ASR回环评估的灾难性失败率），Best-of-N ASR自验证将失败率降至接近零：在标准语料库（LibriSpeech）上N=2时无观测失败，在困难提示集上N=4时亦然。该效果不依赖单一模型，可在四个开源编码器-TTS系统与三种神经编码器（XCodec2、SNAC、Mimi）中复现，其中三个系统在N=2时已达到近零失败水平。随后通过蒸馏将自验证行为注入模型，实现推理时零开销的鲁棒性恢复，在困难输入上关闭约52-58%的失败质量。蒸馏收益集中在高难度输入处；对原本可靠的口语内容无影响。对比实验表明，离线直接偏好优化（DPO/IPO）未优于普通监督蒸馏，而在线迭代变体虽有潜力但统计上未显著区别。我们如实报告了一个难以克服的案例（更大规模的Llasa模型，扩大规模未明显改善），以及所有自蒸馏方法均无法突破的生僻词能力天花板。
原文摘要 · Abstract (English)
Open autoregressive neural-codec text-to-speech (TTS) models sound excellent on typical inputs yet suffer stochastic catastrophic failures: on a meaningful fraction of utterances they emit silence, terminate early, or collapse into repetitive or hallucinated content. We show this failure mode is cheap to remove. Under a single format-robust metric (a catastrophic-failure rate via an ASR round-trip), best-of-N ASR self-verification drives failures to near-zero: no observed failures remain by N=2 on a standard corpus (LibriSpeech) and by N=4 on a hard prompt set. This is not an artifact of one model: the reduction replicates across four open codec-TTS systems and three neural codecs (XCodec2, SNAC, Mimi), reaching the near-zero floor by N=2 on three of the four. We then make the fix free at inference time by distilling the self-verified behaviour into the model, which recovers much of the robustness in single-shot decoding, closing ~52-58% of the failure mass on hard inputs at no test-time cost. The distillation gain concentrates where it is needed (hard inputs); on already-reliable prose there is no headroom and no detectable change. A controlled comparison adds a clean negative: offline direct preference optimization (DPO/IPO) does not beat plain supervised distillation, and an online iterative variant is promising but not statistically separable at our evaluation size. We report honestly the one model that resists (a larger Llasa where scale did not obviously help) and a rare-word capability ceiling that no self-distillation method overcomes
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。