提出语音翻译质量评估新任务，发现端到端方法更有效
SpeechQE: Estimating the Quality of Direct Speech Translation
- 构建语音翻译质量评估任务与基准数据集
- 端到端模型在语音翻译质量预测上优于传统级联方法
- 适合语音翻译、自动评估研究者参考
近年来机器翻译自动质量评估的研究集中于书面语言，语音模态仍被忽视。本文首次提出语音翻译质量评估（SpeechQE）任务，构建基准数据集，并评估基于级联与端到端架构的一系列系统。其中引入一种利用预训练文本大模型的新型端到端方法。结果表明，针对直接语音翻译的质量评估，端到端方法显著优于基于文本质量评估系统的级联方案。研究强调语音翻译质量评估应作为独立问题对待，同时开源数据与模型以推动该领域进一步发展。
原文摘要 · Abstract (English)
Recent advances in automatic quality estimation for machine translation have exclusively focused on written language, leaving the speech modality underexplored. In this work, we formulate the task of quality estimation for speech translation (SpeechQE), construct a benchmark, and evaluate a family of systems based on cascaded and end-to-end architectures. In this process, we introduce a novel end-to-end system leveraging pre-trained text LLM. Results suggest that end-to-end approaches are better suited to estimating the quality of direct speech translation than using quality estimation systems designed for text in cascaded systems. More broadly, we argue that quality estimation of speech translation needs to be studied as a separate problem from that of text, and release our data and models to guide further research in this space.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。