复现NAVER实验室语音指令跟随系统，适配新任务要求并提升翻译与问答性能。
NAVER LABS System Re-implementation for the IWSLT 2026 Instruction-Following Task
- 采用三阶段流程：投影对齐、文本LoRA预训练、多模态融合。
- 在英中语音翻译上达COMET 0.781，英语SQA任务上BERTScore-F1 0.346。
- 构建10万条合成指令数据，覆盖10类语音主导任务，支持后续微调。
我们复现了NAVER LABS在IWSLT 2025的指令跟随流水线，用于IWSLT 2026共享任务（受限条件，短音频）。模型采用SeamlessM4T-v2-large作为语音编码器，Qwen3-4B-Instruct作为大语言模型主干。保留原始设计的三阶段方法：投影对齐、仅文本LoRA预训练、多模态融合。此外，基于提供语料，构建了10万条合成指令跟随样本（每类任务1万条，共10类语音主导任务），可用于第三阶段微调。主模型在英中语音翻译任务上取得COMET 0.781，在MCIF基准的英语SQA任务上达到BERTScore-F1 0.346。
原文摘要 · Abstract (English)
We re-implement the NAVER LABS IWSLT 2025 instruction-following pipeline for the IWSLT 2026 Shared Task (constrained condition, short audio track), adapting it to the mandated components: SeamlessM4T-v2-large as the speech encoder and Qwen3-4B-Instruct as the LLM backbone. The three-stage approach projector alignment, text-only LoRA pre-training, and multimodal merging is preserved from the original design. We additionally construct 100k synthetic instruction-following examples across ten speech-centric task types (10k per task) from the provided corpora, suitable for further Stage 3 fine-tuning. Our primary model achieves COMET 0.781 on EN-ZH speech translation and BERTScore-F1 0.346 on English SQA on the MCIF benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。