端到端语音翻译质量评估模型，无需参考文本即可预测翻译质量。
HydraQE: OSU's Submission for the IWSLT 2026 Speech Translation Metrics Shared Task
- 以音频和译文为输入，通过多层融合与双向Transformer实现跨模态交互。
- 在IWSLT 2026任务中超越级联式文本基基线系统，性能领先3.2 BLEU。
- 适用于缺乏人工标注数据的场景，可利用合成数据与伪标签训练。
我们提出HydraQE，参与IWSLT 2026语音翻译质量评估共享任务。HydraQE是一种基于Qwen3-ASR骨干网络的端到端、无参考质量评估（QE）系统，接收源音频与翻译假设作为联合输入。所有骨干层的隐藏状态通过可学习的稀疏max标量混合进行融合，再经轻量级双向Transformer重新编码，以实现充分的跨模态交互，随后池化为共享嵌入表示。系统采用三个独立预测头，在互补监督信号上进行训练：人类直接评估（DA）标注、MetricX-24伪标签和xCOMET伪标签。为缓解人工标注数据稀缺问题，采用课程学习策略，从合成损坏样本与银色伪标签机器翻译输出开始训练，并逐步过渡至人工标注数据。HydraQE优于级联式文本基基线及先前的直接语音QE系统，表明端到端语音翻译质量评估可与级联方法相媲美。
原文摘要 · Abstract (English)
We present HydraQE, our contribution to the IWSLT 2026 Speech Translation Metrics shared task. HydraQE is an end-to-end, reference-free quality estimation (QE) system for speech translation built on a Qwen3-ASR backbone, which accepts source audio and a translation hypothesis as joint input. Hidden states from all backbone layers are combined via a learnable sparsemax scalar mix, then re-encoded by a lightweight bidirectional Transformer to enable full cross-modal interaction prior to pooling into a shared embedding. Three independent prediction heads are trained on complementary supervision signals: human direct assessment (DA) annotations, MetricX-24 pseudo-labels, and xCOMET pseudo-labels. To address the scarcity of human-annotated data, we train on a combination of synthetically corrupted examples and silver pseudo-labeled machine translation outputs, using a curriculum that begins on synthetic and silver data and gradually shifts toward human-annotated examples. HydraQE outperforms cascaded text-based baselines and prior direct speech QE systems, demonstrating that end-to-end speech translation QE is competitive with cascaded approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。