无需修改模型结构，用数据驱动方法实现低延迟同步语音翻译。
Do LLMs Need Architectural Changes for Simultaneous Speech Translation? A Prefix-to-Prefix Data Driven Approach
- 用固定长度块+回溯机制实现增量解码，不依赖复杂架构。
- 在对话语音上提升1.54分COMETKiwi，延迟仅增加0.15秒。
- 适合追求低延迟、高鲁棒性的实时多语言翻译场景。
同步语音翻译（SimulST）需在严格延迟约束下进行增量翻译，但解码器仅有的大语言模型系统因上下文有限和跨语言重排问题而面临挑战。现有方法常引入架构改动或显式读写策略控制输出时机，但在对话语音中分割边界模糊时易失效。本文提出一种简单数据驱动方案：采用固定长度块进行累积流式解码，并结合基于回溯的已提交前缀，以及带有限等待时间的教师标签前缀到前缀（P2P）目标用于微调，形成CSSEL-P2P，其中CSSEL为提出的分块流式语音编码器大模型。在自建对话语音评估中，CSSEL-P2P相比基线模型提升1.54分COMETKiwi，平均滞后仅增加0.15秒，证明无需架构调整即可实现高效同步语音翻译。
原文摘要 · Abstract (English)
Simultaneous speech translation (SimulST) requires incremental translation under strict latency constraints, yet remains challenging for decoder-only LLM systems due to limited context and cross-lingual reordering. Recent approaches often introduce architectural changes or explicit read/write policies to control output timing, which can be brittle in conversational speech where segmentation boundaries are ambiguous. We present a simple data-driven alternative: fixed-length chunks for cumulative streaming decoding with a rewind-based committed prefix, and teacher-labeled prefix-to-prefix (P2P) targets with bounded waiting for fine-tuning, yielding CSSEL-P2P, where CSSEL is our proposed chunked streaming speech encoder LLM. In our in-house conversational speech evaluation, CSSEL-P2P improves streaming quality by +1.54 COMETKiwi over the CSSEL streaming baseline at comparable latency (+0.15s Average Lagging), suggesting effective SimulST without architectural changes via P2P supervision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。