提升设备端语音翻译速度与质量，实现低延迟实时翻译。
Overcoming Latency Bottlenecks in On-Device Speech Translation: A Cascaded Approach with Alignment-Based Streaming MT
- 分步式架构结合对齐机制，平衡翻译准确率与延迟。
- 采用超时与强制终止策略，降低计算开销，保持实时性。
- 适合移动端实时对话翻译，尤其注重低延迟场景。
本文针对在设备端实现实时流式语音翻译时，自动语音识别（ASR）与机器翻译（MT）融合带来的挑战提出解决方案。尽管基于循环神经网络转换器（RNN-T）的先进ASR系统可实现实时转写，但实现实时流式翻译仍存在显著瓶颈。为此，我们提出一种并行翻译方法，在保证翻译质量的同时有效控制延迟。通过利用ASR生成的语言学线索管理上下文，并采用时间超时与强制终结等高效的束搜索剪枝技术，维持系统的实时因子。我们在设备端双语对话语音翻译任务上验证了该方法，结果表明所提技术在延迟和质量上均优于基线模型。值得注意的是，该方法显著缩小了与非流式翻译系统之间的质量差距，为更精准高效的实时语音翻译铺平道路。
原文摘要 · Abstract (English)
This paper tackles several challenges that arise when integrating Automatic Speech Recognition (ASR) and Machine Translation (MT) for real-time, on-device streaming speech translation. Although state-of-the-art ASR systems based on Recurrent Neural Network Transducers (RNN-T) can perform real-time transcription, achieving streaming translation in real-time remains a significant challenge. To address this issue, we propose a simultaneous translation approach that effectively balances translation quality and latency. We also investigate efficient integration of ASR and MT, leveraging linguistic cues generated by the ASR system to manage context and utilizing efficient beam-search pruning techniques such as time-out and forced finalization to maintain system's real-time factor. We apply our approach to an on-device bilingual conversational speech translation and demonstrate that our techniques outperform baselines in terms of latency and quality. Notably, our technique narrows the quality gap with non-streaming translation systems, paving the way for more accurate and efficient real-time speech translation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。