让语音合成模型学会终身纠错，用记忆存修正不重训。
FlowEdit: Associative Memory for Lifelong Pronunciation Adaptation in Flow-Matching TTS
- 用嵌入空间扰动替代权重更新，实现零成本纠错
- 对312个专有名词的音素错误率降低92.7%
- 15秒内完成修正，适合需要持续优化的语音系统
基于流匹配的文本到语音系统虽具备出色的零样本生成质量，但部署后无法自适应：生僻专有名词的发音错误无法消除，除非重新训练。本文提出FlowEdit，一种针对冻结流匹配语音合成的终身适应框架。该方法将发音修正以潜在条件编辑的形式学习，而非修改模型权重。当收到纠正反馈时，FlowEdit在文本嵌入空间中优化词级扰动，并将修正结果存储于现代霍普菲尔德网络构成的内容可寻址情景记忆中。推理时通过软注意力与相似性门控检索修正，支持模糊形态匹配。在涵盖18个语系的312个多语言专有名词的基准测试中，与零样本基线相比，目标词音素错误率降低92.7%，同时保持原有通用语音质量。单张GPU上修正耗时约15秒。
原文摘要 · Abstract (English)
Flow-matching text-to-speech systems achieve remarkable zero-shot quality but remain static after deployment: pronunciation errors on out-of-vocabulary proper nouns persist unless the model is retrained. We introduce FlowEdit, a life-long adaptation framework for frozen flow-matching TTS that learns pronunciation corrections as latent conditioning edits rather than weight updates. When corrective feedback is provided, FlowEdit optimizes a token-level perturbation in the text embedding space, then stores the correction in a Modern Hopfield Network serving as content-addressable episodic memory. At inference, corrections are retrieved via soft attention with a similarity gate, enabling fuzzy morphological matching. On our curated benchmark of 312 multilingual proper nouns across 18 language families, FlowEdit reduces target-word Phoneme Error Rate by 92.7% relative to the zero-shot baseline while maintaining identical general-speech quality. Corrections complete in approximately 15 seconds on a single GPU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。