一个系统同时完成语音识别、翻译和问答，支持英到中、意、德的多语言处理。
NAVER LABS Europe Submission to the Instruction-following Track
- 用语音转大模型嵌入投影+LoRA微调的双模块架构。
- 在1000步指令微调后，实现英语语音到多语言的端到端任务。
- 适合需要多任务语音处理的工业级应用开发者。
本文描述了NAVER LABS Europe参加IWSLT 2025指令跟随语音处理短赛道的提交方案。在受限设置下，我们开发了能从英语语音输入中同时执行语音识别（ASR）、语音翻译（ST）和语音问答（SQA）的系统，目标语言包括中文、意大利语和德语。我们的解决方案基于两个预训练模块：(1) 使用SeamlessM4T-v2-large语音编码器表示训练的语音到大模型嵌入投影器；(2) 在Llama-3.1-8B-Instruct基础上，使用文本数据训练的LoRA适配器。这两个模块联合加载，并在多语言多模态数据上进行1000步指令微调，形成最终提交评估的系统。
原文摘要 · Abstract (English)
In this paper we describe NAVER LABS Europe submission to the instruction-following speech processing short track at IWSLT 2025. We participate in the constrained settings, developing systems that can simultaneously perform ASR, ST, and SQA tasks from English speech input into the following target languages: Chinese, Italian, and German. Our solution leverages two pretrained modules: (1) a speech-to-LLM embedding projector trained using representations from the SeamlessM4T-v2-large speech encoder; and (2) LoRA adapters trained on text data on top of a Llama-3.1-8B-Instruct. These modules are jointly loaded and further instruction-tuned for 1K steps on multilingual and multimodal data to form our final system submitted for evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。