在普通微控制器上实现最小实时神经语音合成，仅需67万字节存储。
sanoTTS: The Smallest Real-Time Neural TTS on a General-Purpose Microcontroller
- 基于条件变分自编码器蒸馏，将大模型压缩为56万参数小模型。
- ESP32-S3上实时生成4.54秒语音仅用1.02秒，达0.22倍实时速度。
- 适合嵌入式设备部署，尤其对低功耗、小体积语音应用有价值。
本文介绍了一个可在通用微控制器上运行的神经文本转语音系统，从音素ID生成22.05 kHz PCM波形。其部署图仅含567,008个参数，两个int8数据块共占679,832字节。在ESP32-S3上，完整时长-声学-逆STFT路径生成4.54秒语音仅耗时1.02秒（0.22倍实时），无需神经加速器。同一套便携C核心在无浮点单元的ESP32-C3上离线运行可达5.72倍实时。据我们所知，这是首个在无神经加速器的通用微控制器上实现实时端到端神经语音合成且模型最小的方案。模型通过从Piper/VITS教师模型中蒸馏获得，训练使用了时长、潜在接口、波形、对抗及联合蒸馏损失。性能有所下降：在未见文本上，从en_US-kristin-medium蒸馏的模型得分2.54 SCOREQ和2.80 UTMOS，而教师模型为4.68和4.42。另有一独立英语质量包采用更强的en_US-amy-medium教师，其1,454,284参数版本得分为4.13 SCOREQ和4.10 UTMOS；1,834,380参数版本得分为4.16 SCOREQ。容量研究显示，解码器是主要瓶颈而非输出表示。两次评估失误：测试集过于模板化导致早期学生模型SCOREQ虚高1.35分；聚合质量预测未能发现听感明显异常的咝音问题，该问题在逐音素谱探针中可识别。所有模型块、运行时端口和黄金向量均提供校验和。
原文摘要 · Abstract (English)
This paper describes an audited neural text-to-speech stack that runs from phoneme IDs to 22.05-kHz PCM on general-purpose microcontrollers. Its deployed graph has 567,008 parameters, and its two int8 blobs occupy 679,832 bytes. On an ESP32-S3, the complete duration-acoustic-inverse-STFT path generates 4.54 s of speech in 1.02 s (0.22x real time) without a neural accelerator. The same portable C core runs offline at 5.72x real time on an FPU-less ESP32-C3. To our knowledge, this is the smallest complete phoneme-to-waveform neural TTS graph demonstrated in real time on a general-purpose microcontroller without a neural accelerator. We derive the students from the conditional-VAE objective of their Piper/VITS teachers and state the duration, latent-interface, waveform, adversarial, and joint-distillation losses used in training. The size and speed come with an audible cost: on unseen text, the embedded stack distilled from en_US-kristin-medium scores 2.54 SCOREQ and 2.80 UTMOS, compared with 4.68 and 4.42 for its teacher. A separate English quality package uses the stronger en_US-amy-medium teacher. Its 1,454,284-parameter Pareto point scores 4.13 SCOREQ and 4.10 UTMOS; a 1,834,380-parameter variant scores 4.16 SCOREQ. A controlled capacity study with Kristin identifies the decoder, rather than the output representation, as the main constraint. Two evaluation failures also affected the work: a narrow, templated test set overstated one early student's SCOREQ by 1.35, and aggregate quality predictors missed a sibilant failure that was evident in listening and in a phoneme-resolved spectral probe. Checksums cover the reported model blobs, runtime ports, and golden vectors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。