通过嵌入空间混合法生成新语音类别，提升说话人验证泛化能力。
CAARMA: Class Augmentation with Adversarial Mixup Regularization
- 在嵌入空间混合数据生成合成类别，扩充训练集多样性。
- 对抗性优化使合成类别与真实类别区分度降低，提升真实性。
- 在多个零样本语音任务中提升8%性能，适合少样本语音识别场景。
说话人验证是典型的零样本学习任务，需将测试样本的嵌入向量与已知样本比较以推断类别。模型必须生成紧凑聚类的同类别嵌入，同时保持跨类别分离。通常通过大量类别（说话人）训练并使用特定损失函数实现，但真实语音数据集常缺乏足够的类别多样性，影响泛化能力。本文提出CAARMA，一种类别增强框架，通过在嵌入空间进行数据混合生成合成类别，扩展训练类别数量。为确保合成类别的真实性，引入新颖的对抗性精炼机制，最小化合成类别与真实类别间的分类差异。在多个说话人验证任务及其他代表性零样本语音分析任务上评估，结果一致显示优于所有基线模型8%。代码已开源。
原文摘要 · Abstract (English)
Speaker verification is a typical zero-shot learning task, where inference of unseen classes is performed by comparing embeddings of test instances to known examples. The models performing inference must hence naturally generate embeddings that cluster same-class instances compactly, while maintaining separation across classes. In order to learn to do so, they are typically trained on a large number of classes (speakers), often using specialized losses. However real-world speaker datasets often lack the class diversity needed to effectively learn this in a generalizable manner. We introduce CAARMA, a class augmentation framework that addresses this problem by generating synthetic classes through data mixing in the embedding space, expanding the number of training classes. To ensure the authenticity of the synthetic classes we adopt a novel adversarial refinement mechanism that minimizes categorical distinctions between synthetic and real classes. We evaluate CAARMA on multiple speaker verification tasks, as well as other representative zero-shot comparison-based speech analysis tasks and obtain consistent improvements: our framework demonstrates a significant improvement of 8\% over all baseline models. The code is available at: https://github.com/massabaali7/CAARMA/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。