解决流式生成中非单调对齐难题，提升翻译等任务性能
Overcoming Non-monotonicity in Transducer-based Streaming Generation
- 引入可学习单调注意力，融合输入历史信息改进解码
- 通过前向后向算法高效推断对齐概率，避免指数级计算
- 适用于需要非单调对齐的复杂流式任务，如同步翻译
流式生成模型广泛应用于多个领域，其中基于转换器（Transducer）的架构在工业应用中尤为流行。然而，其输入同步解码机制在需要非单调对齐的任务（如同步翻译）中面临挑战。本文提出将转换器解码与输入流历史相结合，引入可学习的单调注意力机制。该方法利用前向-后向算法推断预测状态与输入时间戳之间的对齐后验概率，并据此估计单调上下文表示，从而避免训练时枚举指数级对齐空间。大量实验表明，所提出的MonoAttn-Transducer在流式场景中有效处理非单调对齐问题，为复杂生成任务提供稳健解决方案。
原文摘要 · Abstract (English)
Streaming generation models are utilized across fields, with the Transducer architecture being popular in industrial applications. However, its input-synchronous decoding mechanism presents challenges in tasks requiring non-monotonic alignments, such as simultaneous translation. In this research, we address this issue by integrating Transducer's decoding with the history of input stream via a learnable monotonic attention. Our approach leverages the forward-backward algorithm to infer the posterior probability of alignments between the predictor states and input timestamps, which is then used to estimate the monotonic context representations, thereby avoiding the need to enumerate the exponentially large alignment space during training. Extensive experiments show that our MonoAttn-Transducer effectively handles non-monotonic alignments in streaming scenarios, offering a robust solution for complex generation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。