用模拟对话训练模型，30分钟真实数据即可高效实现儿童成人语音分离。
Data Efficient Child-Adult Speaker Diarization with Simulated Conversations
- 用AudioSet生成儿童与成人对话的模拟数据，解决真实标注数据稀缺问题。
- 仅用30分钟真实数据微调，模型在真实数据上表现显著提升。
- 适合需要低成本构建儿童语音分析系统的研究者和开发者。
自动化儿童语音分析对神经认知评估等应用至关重要。说话人分离（即识别谁在何时说话）是其关键环节。然而，由于隐私问题和缺乏标注数据集，公开可用的儿童-成人说话人分离方案极为稀少，而为每种场景手动标注数据又耗时且昂贵。为此，我们提出一种数据高效解决方案：利用AudioSet生成模拟的儿童-成人对话，训练基于Whisper Encoder的模型。该模型在真实数据集上表现出强大的零样本性能，仅需30分钟真实数据微调即可显著提升效果，使用LoRA进一步优化了迁移学习性能。源代码及在模拟数据上训练的模型已公开。
原文摘要 · Abstract (English)
Automating child speech analysis is crucial for applications such as neurocognitive assessments. Speaker diarization, which identifies ``who spoke when'', is an essential component of the automated analysis. However, publicly available child-adult speaker diarization solutions are scarce due to privacy concerns and a lack of annotated datasets, while manually annotating data for each scenario is both time-consuming and costly. To overcome these challenges, we propose a data-efficient solution by creating simulated child-adult conversations using AudioSet. We then train a Whisper Encoder-based model, achieving strong zero-shot performance on child-adult speaker diarization using real datasets. The model performance improves substantially when fine-tuned with only 30 minutes of real train data, with LoRA further improving the transfer learning performance. The source code and the child-adult speaker diarization model trained on simulated conversations are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。