用语音语言模型实现无需标注的多风格语音分割。
Unsupervised Speech Segmentation: A General Approach Using Speech Language Models
- 基于语音语言模型，无监督识别语音中的多类声学语义变化。
- 在边界检测与段落纯度上优于基线方法，减少过度分割。
- 适合需要自动识别情绪、说话人等非文本特征的应用场景。
本文提出一种无监督语音分割通用方法，基于语音语言模型（SLMs），不仅适用于传统语音分割任务，还能识别多种声学-语义风格变化，如情感、说话人等难以转化为文本的特征。不同于仅关注频谱变化的传统方法，该方法聚焦于语音中声学语义差异的划分。实验表明，所提方法在边界检测、段落纯度和过分割控制方面均优于多个基线模型。代码已开源。
原文摘要 · Abstract (English)
In this paper, we introduce an unsupervised approach for Speech Segmentation, which builds on previously researched approaches, e.g., Speaker Diarization, while being applicable to an inclusive set of acoustic-semantic distinctions, paving a path towards a general Unsupervised Speech Segmentation approach. Unlike traditional speech and audio segmentation, which mainly focuses on spectral changes in the input signal, e.g., phone segmentation, our approach tries to segment the spoken utterance into chunks with differing acoustic-semantic styles, focusing on acoustic-semantic information that does not translate well into text, e.g., emotion or speaker. While most Speech Segmentation tasks only handle one style change, e.g., emotion diarization, our approach tries to handle multiple acoustic-semantic style changes. Leveraging recent advances in Speech Language Models (SLMs), we propose a simple unsupervised method to segment a given speech utterance. We empirically demonstrate the effectiveness of the proposed approach by considering several setups. Results suggest that the proposed method is superior to the evaluated baselines on boundary detection, segment purity, and over-segmentation. Code is available at https://github.com/avishaiElmakies/unsupervised_speech_segmentation_using_slm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。