研究帧率对中英文语音分词的影响，发现不同语言响应差异。
Impact of Frame Rates on Speech Tokenizer: A Case Study on Mandarin and English
- 通过不同帧率编码语音，分析其对分词结果的影响
- 中英文在帧率变化下表现不同，受语音密度和音系特征影响
- 为语音识别等应用的帧率选择提供依据
语音分词器在现代语音任务中扮演关键角色，通常作为语音信号与语言模型之间的桥梁。尽管低帧率编码器被广泛用作语音分词器，但帧率对语音分词的影响仍缺乏深入研究。本文以语言类型迥异的中文和英文为例，考察不同帧率下语音编码对分词效果的影响，并在语音识别任务中评估生成的语义分词。结果表明，帧率变化对两种语言的分词性能产生差异化影响，凸显了帧率、语音音素密度与语言特异性声学特征之间的相互作用。研究为语音分词器的帧率优化提供了重要启示，对自动语音识别、文本转语音等应用具有实际意义。
原文摘要 · Abstract (English)
The speech tokenizer plays a crucial role in recent speech tasks, generally serving as a bridge between speech signals and language models. While low-frame-rate codecs are widely employed as speech tokenizers, the impact of frame rates on speech tokens remains underexplored. In this study, we investigate how varying frame rates affect speech tokenization by examining Mandarin and English, two typologically distinct languages. We encode speech at different frame rates and evaluate the resulting semantic tokens in the speech recognition task. Our findings reveal that frame rate variations influence speech tokenization differently for each language, highlighting the interplay between frame rates, phonetic density, and language-specific acoustic features. The results provide insights into optimizing frame rate selection for speech tokenizers, with implications for automatic speech recognition, text-to-speech, and other speech-related applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。