提出新评估方法，提升多语言大模型的语种遵循能力
CoCo-CoLa: Evaluating and Improving Language Adherence in Multilingual LLMs
- 设计新指标CoCo-CoLa，量化模型输出语言准确性
- 发现模型最终层决定语种选择，低资源语言表现更差
- 仅微调关键层即可显著提升低资源语言表现，节省算力
多语言大模型虽在有限平行数据下具备跨语言能力，但常倾向于生成高资源语言（如英语）响应。本文提出CoCo-CoLa（正确概念-正确语言）评估指标，通过在七种语言的闭卷问答任务上进行微调实验，分析单语训练对其他语言的影响。结果表明，模型共享任务知识但存在输出语言偏见；识别出语言特定层，发现最终层在决定输出语言中起关键作用。据此提出部分微调策略，仅针对关键层进行训练，在显著降低计算成本的同时，实现与全量微调相当或更优的性能，尤其改善低资源语言表现，提供更高效的多语言适配方案。
原文摘要 · Abstract (English)
Multilingual Large Language Models (LLMs) develop cross-lingual abilities despite being trained on limited parallel data. However, they often struggle to generate responses in the intended language, favoring high-resource languages such as English. In this work, we introduce CoCo-CoLa (Correct Concept - Correct Language), a novel metric to evaluate language adherence in multilingual LLMs. Using fine-tuning experiments on a closed-book QA task across seven languages, we analyze how training in one language affects others' performance. Our findings reveal that multilingual models share task knowledge across languages but exhibit biases in the selection of output language. We identify language-specific layers, showing that final layers play a crucial role in determining output language. Accordingly, we propose a partial training strategy that selectively fine-tunes key layers, improving language adherence while significantly reducing computational cost. Our method achieves comparable or superior performance to full fine-tuning, particularly for low-resource languages, offering a more efficient multilingual adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。