用极少量数据蒸馏大模型，打造能在笔记本上实时运行的高质印地语语音合成器
Staged Depth-Pruning Distillation of a Flow-Matching Text-to-Speech Teacher: A Compact Hindi Speech Synthesizer
- 仅剪枝深度保留教师模型结构，逐步瘦身并用语音识别错误率监控
- 190M参数模型在未见语句上实现0%语音识别错误率，最低至131M仍稳定
- 适合资源受限场景下的本地化语音合成部署，尤其对印地语支持友好
我们提出一种实用方法，基于严重数据限制（约17.6小时）构建紧凑的印地语文本转语音（TTS）模型。通过仅剪枝深度的方式，从一个大型流匹配教师模型（IndicF5，337M参数，DiT结构）中蒸馏知识：保持宽度、文本维度、注意力头数和输入输出不变，仅保留均匀分布的子集变换块。先测试教师模型耐受度（删去27%块仍可用，超50%则崩溃），再分步递减（22→16→12→8→6块），每步后微调并以语音识别词错误率（WER）为约束。最终学生模型在249M和190M参数下达到0.00 WER，且稳定至131M；102M时出现性能断崖，归因于数据预算而非方法。还发现两个训练/推理不一致问题（梅尔滤波器组与旋转嵌入库版本差异），导致音频质量无声下降，并提供版本无关修复方案。该方法生成高质量印地语语音，可在6GB显存笔记本GPU上实时运行。独立50句FLEURS基准对比显示，发布的190M学生模型优于其教师与MMS-TTS-hin。
原文摘要 · Abstract (English)
We present a practical recipe for building a compact Hindi text-to-speech (TTS) model by distilling a large flow-matching teacher (IndicF5, 337M-parameter DiT) under a severe data budget (~17.6 hours). Training a small model from scratch on this much data fails outright. Instead we warm-start the student from the teacher by pruning depth only: keeping the teacher's width, text dimension, attention heads, and mel/text I/O fixed so all non-block tensors copy one-to-one, and retaining an evenly-spaced subset of transformer blocks. We first measure how much depth the teacher tolerates (it remains near-functional at -27% blocks but collapses past -50%), then descend gradually (22 -> 16 -> 12 -> 8 -> 6 blocks), re-fine-tuning after each prune, with each step gated by an objective ASR word-error-rate (WER) check. The resulting students reach WER 0.00 on unseen sentences at 249M and 190M parameters, and remain robust down to 131M; at 102M we observe a clear capacity cliff that we attribute to the data budget rather than the recipe. We also document two train/inference feature- and library-parity failures (mel filterbank and rotary-embedding library versions) that silently degrade audio, and a version-independent fix. The method yields a high-quality Hindi voice that runs in real time on a 6 GB laptop GPU. An independent 50-sentence FLEURS benchmark compares the released 190M student against its teacher and MMS-TTS-hin.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。