无需训练，用自注意力实现长文本实时翻译
DOA: Training-Free Decoder-Only Attention Policy for Long-Form Simultaneous Translation with SpeechLLMs
- 从自注意力中提取代理对齐信号，构建免训练的推理策略
- 在Phi4-Multimodal和Qwen3-Omni上实现低延迟长文本翻译
- 适合希望直接部署SpeechLLM做实时翻译的研究者
同时语音转写翻译（SimulST）在语音未结束时即生成译文，需依赖流式策略决定读取与输出时机。现有先进方法多采用基于注意力的编码器-解码器模型，通过交叉注意力提供显式对齐信号。而语音大语言模型（SpeechLLMs）为仅解码器架构，仅依赖自注意力。这引发核心问题：自注意力能否提供足够稳定的对齐信号以指导流式策略？此外，现有方法通常依赖训练适配或启发式wait-$k$策略，且未在长文本场景验证。为此，本文提出无需训练的解码器自注意力（DOA）策略，通过自注意力提取代理对齐信号，使现成SpeechLLM即可实现长文本同时翻译。在Phi4-Multimodal和Qwen3-Omni上的实验表明，DOA能有效支撑流式决策，在不重新训练的前提下实现接近离线解码质量的低延迟长文本翻译。
原文摘要 · Abstract (English)
Simultaneous speech-to-text translation (SimulST) generates translations while speech is still unfolding, requiring a streaming policy that decides when to read and when to write. State-of-the-art approaches rely on attention-based encoder-decoder models where cross-attention provides explicit alignment signals. In contrast, Speech Large Language Models (SpeechLLMs) are decoder-only architectures relying solely on self-attention. This raises a central question: whether decoder self-attention contains sufficiently stable alignment signals to guide the streaming policy. Moreover, existing approaches typically rely on training-based adaptations or heuristic wait-$k$ policies and have not been validated in long-form settings. To fill these gaps, we propose Decoder-Only Attention (DOA), a training-free policy that enables long-form simultaneous translation with off-the-shelf SpeechLLMs by deriving a proxy alignment from self-attention. Experiments on Phi4-Multimodal and Qwen3-Omni show that DOA provides an effective alignment signal for supporting streaming decisions, enabling low-latency long-form SimulST with quality close to offline decoding without retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。