无需训练，直接从冻结的WavLM提取音节边界与嵌入，性能超越现有方法。
ZeroSyl: Simple Zero-Resource Syllable Tokenization for Spoken Language Modeling
- 利用WavLM中间层特征的L2范数自动定位音节边界
- 在词汇、句法和叙事任务上均优于已有音节分词器
- 适合追求简洁高效语音建模的开发者与研究者
纯语音语言模型旨在直接从原始音频中学习语言，而无需文本资源。关键挑战在于自监督语音编码器生成的离散标记序列过长，促使近期研究转向音节类单元。然而，如Sylber和SyllableLM等方法依赖复杂的多阶段训练流程。本文提出ZeroSyl，一种无需训练的方法，可直接从冻结的WavLM模型中提取音节边界与嵌入。通过分析WavLM中间层特征的L2范数，实现具有竞争力的音节分割效果。所得片段经均值池化后，使用K-means进行离散化，并用于语言模型训练。实验表明，ZeroSyl在词汇、句法和叙事基准测试中均优于先前的音节分词器。扩展性实验显示，尽管更细粒度的单元对词汇任务有益，但本方法发现的音节单元在句法建模方面表现出更优的可扩展性。
原文摘要 · Abstract (English)
Pure speech language models aim to learn language directly from raw audio without textual resources. A key challenge is that discrete tokens from self-supervised speech encoders result in excessively long sequences, motivating recent work on syllable-like units. However, methods like Sylber and SyllableLM rely on intricate multi-stage training pipelines. We propose ZeroSyl, a simple training-free method to extract syllable boundaries and embeddings directly from a frozen WavLM model. Using L2 norms of features in WavLM's intermediate layers, ZeroSyl achieves competitive syllable segmentation performance. The resulting segments are mean-pooled, discretized using K-means, and used to train a language model. ZeroSyl outperforms prior syllabic tokenizers across lexical, syntactic, and narrative benchmarks. Scaling experiments show that while finer-grained units are beneficial for lexical tasks, our discovered syllabic units exhibit better scaling behavior for syntactic modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。