让语音模型动态激活任务，支持部分标注数据训练。
TokenVerse++: Towards Flexible Multitask Learning with Dynamic Task Activation
- 在声学嵌入空间引入可学习向量，实现任务动态激活
- 仅需部分标签即可训练，提升对不完整标注数据的利用率
- 兼容多任务且不影响语音识别性能，适合实际场景应用
基于标记的多任务框架如 TokenVerse 要求所有训练语句对所有任务都有标签，限制了其对部分标注数据的利用和扩展能力。本文提出 TokenVerse++，在 XLSR-Transducer ASR 模型的声学嵌入空间中引入可学习向量，实现动态任务激活。该机制使模型能够使用仅部分任务有标签的语句进行训练，显著提升数据利用灵活性。我们通过将一个包含部分标签的数据集（用于语音识别与语言识别）成功整合进训练流程，验证了该方法的有效性。TokenVerse++ 在多个任务上的表现与或优于 TokenVerse，同时保持了与原始模型相当的语音识别性能，证明其是更实用的多任务学习方案。
原文摘要 · Abstract (English)
Token-based multitasking frameworks like TokenVerse require all training utterances to have labels for all tasks, hindering their ability to leverage partially annotated datasets and scale effectively. We propose TokenVerse++, which introduces learnable vectors in the acoustic embedding space of the XLSR-Transducer ASR model for dynamic task activation. This core mechanism enables training with utterances labeled for only a subset of tasks, a key advantage over TokenVerse. We demonstrate this by successfully integrating a dataset with partial labels, specifically for ASR and an additional task, language identification, improving overall performance. TokenVerse++ achieves results on par with or exceeding TokenVerse across multiple tasks, establishing it as a more practical multitask alternative without sacrificing ASR performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。