用音频大模型自动筛选语音翻译训练数据，提升翻译质量。
Leveraging Audio-LLMs to Filter Speech-to-Speech Training Data
- 用轻量级排序器生成伪标签，训练音频大模型直接判断语音对是否保留。
- 在两个数据集上实验，端到端语音翻译性能最高提升1.4 ASR-BLEU。
- 适合需要高质量语音翻译训练数据的研究者和开发者。
大规模挖掘语料为端到端语音到语音翻译（S2ST）提供了丰富的训练数据，但可能包含噪声、对齐错误和语义错误。过滤噪声数据对保持语音翻译的鲁棒性至关重要。本文研究如何训练一个音频语言模型，直接从音频对中做出保留/丢弃决策。为获得可靠监督信号而无需人工标注，我们采用可扩展的两阶段「排序到蒸馏」策略：先用轻量级排序器从噪声语音对生成保留/丢弃伪标签，再训练音频大语言模型直接从原始语音对预测保留/丢弃。该模型联合捕捉声学保真度与跨语言语义一致性，用于选择语音条件数据。在CVSS-C和SpeechMatrix上的实验表明，相比未过滤训练，性能持续提升，端到端S2ST最高达+1.4 ASR-BLEU。
原文摘要 · Abstract (English)
Large-scale mined corpora provide abundant training data for end-to-end speech-to-speech translation (S2ST) but may contain noise, misalignment, and semantic errors. Filtering noisy data is crucial to maintain robust speech translation performance. We study how to train an audio-language model to make keep/drop decisions on paired speech directly from audio. To obtain reliable supervision without manual labels, we adopt a scalable two-stage Rank-to-Distill strategy. A lightweight ranker generates keep/drop pseudo-labels from noisy speech pairs, then trains an audio large language model to predict keep/drop directly from raw paired speech. The resulting model jointly captures acoustic fidelity and cross-lingual semantic consistency for the selection of speech-conditioned data. Experiments on CVSS-C and SpeechMatrix show consistent improvements over unfiltered training, yielding up to +1.4 ASR-BLEU for end-to-end S2ST.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。