提出一种评估长篇实时语音翻译的方法，解决现有评估不真实、难复现的问题。
A Practical Evaluation Method for Long-Form Simultaneous Speech-to-Speech Translation
- 用ASR和强制对齐恢复目标语音的词级时间戳，实现句级评估
- 发现当前系统在长语音上存在严重延迟累积现象
- 适合研究实时语音翻译系统的开发者和评估者使用
同时语音到语音翻译（SimulS2ST）支持实时跨语言交流，但现有评估多集中于短时或预分段语音，而非连续长篇输入。以往方法难以复现，且对端到端系统假设不成立。本文提出一种实用的长篇SimulS2ST评估方法：给定源语音、预分段源文本和参考译文，对生成的目标语音进行自动语音识别（ASR）和强制对齐，恢复词级时间戳，再用句向量对齐器将目标文本匹配到对应源句。由此可计算句级延迟与质量指标（如YAAL、xCOMET），并聚合为系统级评分。在多个代表性SimulS2ST系统上的实验表明该方法有效，揭示当前系统在长语音上存在显著延迟累积问题。代码已公开：https://github.com/SakaiXue6666/Speech-to-Speech-Latency
原文摘要 · Abstract (English)
Simultaneous speech-to-speech translation (SimulS2ST) enables real-time cross-lingual communication, but existing evaluation has focused largely on short or pre-segmented speech rather than long-form, continuous input. Prior approaches are difficult to reproduce and make assumptions that do not hold for end-to-end systems. We present a practical evaluation method for long-form SimulS2ST. Given source speech, pre-segmented source transcripts, and reference translations, we run automatic speech recognition (ASR) and forced alignment on the generated target speech to recover token-level timestamps, then apply a sentence-embedding-based aligner to match the target text to its corresponding source sentences. This enables sentence-level computation of latency and quality metrics, including YAAL and xCOMET, which are then aggregated into final system-level scores. Experiments on representative SimulS2ST systems show that the method is effective in practice and reveal that current systems suffer from substantial latency accumulation on long speech. Code can be found here https://github.com/SakaiXue6666/Speech-to-Speech-Latency
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。