用端到端语音编码直接分段新闻,跨语言效果更好
Advancing Topic Segmentation of Broadcasted Speech with Multilingual Semantic Embeddings
- 直接用语音编码器做话题分段,跳过转录文本步骤
- 英语上达0.2564的分割精度,多语言训练后提升至0.2370
- 首个跨语种零样本测试的广播新闻分段基准数据集
近期语音话题分段研究显示,预训练语音编码器可直接从语音中捕捉语义表征。传统方法依赖两步流程:先生成语音识别转录文本,再进行文本分段。本文提出一种端到端方案,直接使用语义语音编码器完成分段,避免传统流程。聚焦广播新闻领域——单段录音中存在多样说话人与话题,带来挑战——我们实现了高效端到端的话题切变点检测。此外,我们构建了一个新基准数据集,包含约1000小时六种欧洲语言的公开录音,并加入印地语评估集,用于测试模型在跨语言、零样本场景下的表现。该设置反映真实世界多样性及模型适应多种语言的需求。结果表明,尽管传统流水线方法在英语上达到0.2431的P_k分数,我们的端到端模型达到0.2564,多语言训练后分别提升至0.1988和0.2370。为支持后续研究,我们发布模型及数据处理脚本,推动多语言广播新闻话题分段的开放研究。
原文摘要 · Abstract (English)
Recent advancements in speech-based topic segmentation have highlighted the potential of pretrained speech encoders to capture semantic representations directly from speech. Traditionally, topic segmentation has relied on a pipeline approach in which transcripts of the automatic speech recognition systems are generated, followed by text-based segmentation algorithms. In this paper, we introduce an end-to-end scheme that bypasses this conventional two-step process by directly employing semantic speech encoders for segmentation. Focused on the broadcasted news domain, which poses unique challenges due to the diversity of speakers and topics within single recordings, we address the challenge of accessing topic change points efficiently in an end-to-end manner. Furthermore, we propose a new benchmark for spoken news topic segmentation by utilizing a dataset featuring approximately 1000 hours of publicly available recordings across six European languages and including an evaluation set in Hindi to test the model's cross-domain performance in a cross-lingual, zero-shot scenario. This setup reflects real-world diversity and the need for models adapting to various linguistic settings. Our results demonstrate that while the traditional pipeline approach achieves a state-of-the-art $P_k$ score of 0.2431 for English, our end-to-end model delivers a competitive $P_k$ score of 0.2564. When trained multilingually, these scores further improve to 0.1988 and 0.2370, respectively. To support further research, we release our model along with data preparation scripts, facilitating open research on multilingual spoken news topic segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。