直接语音翻译比思维链提示在数据增多时表现更好
Revisiting Direct Speech-to-Text Translation with Speech LLMs: Better Scaling than CoT Prompting?
- 用伪标签构建多语言语音翻译数据集，对比直接与思维链提示
- 数据量越大，直接提示的性能提升越明显，超越思维链方法
- 适合关注大模型语音翻译效率与数据扩展性的研究者
近期语音到文本翻译（S2TT）研究聚焦于大语言模型（LLM），普遍采用思维链（CoT）提示，即先转录语音再翻译文本。该方法通常优于直接提示，因其可利用大量语音识别（ASR）和文本翻译（T2TT）数据显式建模步骤。本文系统比较了不同数据规模下CoT与直接提示的表现。为此，我们通过将一个ASR语料库的转录文本伪标注为六种欧洲语言，构建了大规模伪标签S2TT数据集，并在不同数据量下训练基于LLM的S2TT系统。结果表明，随着数据量增加，直接提示的性能提升更稳定、更显著，暗示未来更大规模的S2TT资源下，直接提示可能成为更有效的方案。
原文摘要 · Abstract (English)
Recent work on Speech-to-Text Translation (S2TT) has focused on LLM-based models, introducing the increasingly adopted Chain-of-Thought (CoT) prompting, where the model is guided to first transcribe the speech and then translate it. CoT typically outperforms direct prompting primarily because it can exploit abundant Automatic Speech Recognition (ASR) and Text-to-Text Translation (T2TT) datasets to explicitly model its steps. In this paper, we systematically compare CoT and Direct prompting under increasing amounts of S2TT data. To this end, we pseudo-label an ASR corpus by translating its transcriptions into six European languages, and train LLM-based S2TT systems with both prompting strategies at different data scales. Our results show that Direct improves more consistently as the amount of data increases, suggesting that it may become a more effective approach as larger S2TT resources are created.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。