用最优传输设计可微对齐框架，提升语音识别序列对齐精度
A Differentiable Alignment Framework for Sequence-to-Sequence Modeling via Optimal Transport
- 基于一维最优传输构建可微对齐机制
- 在TIMIT/AMI/LibriSpeech上对齐性能优于CTC
- 适合需要精确时序对齐的研究与应用
准确的序列到序列（seq2seq）对齐对于医疗语音分析和语言学习工具等依赖自动语音识别（ASR）的应用至关重要。当前主流端到端（E2E）ASR系统如连接时序分类（CTC）和基于转换器的模型存在峰值行为和对齐不准确问题。本文提出一种基于一维最优传输的新型可微对齐框架，使模型能够以端到端方式学习单一对齐。我们引入一种称为序列最优传输距离（SOTD）的伪度量，并讨论其理论性质。基于SOTD，提出最优时间传输分类（OTTC）损失用于ASR，对比其与CTC的行为差异。在TIMIT、AMI和LibriSpeech数据集上的实验表明，该方法显著提升了对齐性能，优于CTC及近期提出的正则化一致性CTC，尽管在ASR性能上存在一定权衡。我们认为这项工作为seq2seq对齐研究开辟了新方向，为社区后续探索提供了坚实基础。代码公开于：https://github.com/idiap/OTTC
原文摘要 · Abstract (English)
Accurate sequence-to-sequence (seq2seq) alignment is critical for applications like medical speech analysis and language learning tools relying on automatic speech recognition (ASR). State-of-the-art end-to-end (E2E) ASR systems, such as the Connectionist Temporal Classification (CTC) and transducer-based models, suffer from peaky behavior and alignment inaccuracies. In this paper, we propose a novel differentiable alignment framework based on one-dimensional optimal transport, enabling the model to learn a single alignment and perform ASR in an E2E manner. We introduce a pseudo-metric, called Sequence Optimal Transport Distance (SOTD), over the sequence space and discuss its theoretical properties. Based on the SOTD, we propose Optimal Temporal Transport Classification (OTTC) loss for ASR and contrast its behavior with CTC. Experimental results on the TIMIT, AMI, and LibriSpeech datasets show that our method considerably improves alignment performance compared to CTC and the more recently proposed Consistency-Regularized CTC, though with a trade-off in ASR performance. We believe this work opens new avenues for seq2seq alignment research, providing a solid foundation for further exploration and development within the community. Our code is publicly available at: https://github.com/idiap/OTTC
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。