arXiv:2511.09984cs.CL2025-11被引 8

多语言检索生成中,模型常因解码机制偏差而输出错误语言,本文提出轻量级修正方法。

Language Drift in Multilingual Retrieval-Augmented Generation: Characterization and Decoding-Time Mitigation

  • 通过惩罚非目标语言词元,温和引导生成方向
  • 在三个数据集上显著提升多语言一致性与任务准确率
  • 无需训练、适配任意模型,适合多语言应用开发者

多语言检索增强生成(RAG)利用检索文档作为外部证据,使大语言模型在多语言场景下完成知识密集型任务。然而,当检索内容与用户查询及上下文示例语言不一致时,模型常出现语言漂移,生成非预期语言的回应。这种现象在链式思维(CoT)等推理密集型解码过程中尤为明显,中间步骤加剧语言不稳定性。本研究系统分析了多语言RAG中的输出语言漂移问题,覆盖多个数据集、语言和模型架构。受控实验表明,漂移并非理解失败所致,而是解码器层面的分布坍塌——主流词元分布和高频英语模式主导了生成结果。进一步发现,在跨语言条件下,英语成为语义吸引子,既是主要干扰源,也是最常见的回退语言。为此,我们提出软约束解码(SCD),一种无需训练、模型无关的轻量级解码策略,通过惩罚非目标语言词元,温和引导生成。SCD可无缝集成至任何生成算法,无需修改架构或额外数据。在三个多语言数据集和多种类型语言上的实验显示,SCD持续提升语言对齐度与任务性能,为多语言RAG提供高效通用的解决方案。

原文摘要 · Abstract (English)

Multilingual Retrieval-Augmented Generation (RAG) enables large language models (LLMs) to perform knowledge-intensive tasks in multilingual settings by leveraging retrieved documents as external evidence. However, when the retrieved evidence differs in language from the user query and in-context exemplars, the model often exhibits language drift by generating responses in an unintended language. This phenomenon is especially pronounced during reasoning-intensive decoding, such as Chain-of-Thought (CoT) generation, where intermediate steps introduce further language instability. In this paper, we systematically study output language drift in multilingual RAG across multiple datasets, languages, and LLM backbones. Our controlled experiments reveal that the drift results not from comprehension failure but from decoder-level collapse, where dominant token distributions and high-frequency English patterns dominate the intended generation language. We further observe that English serves as a semantic attractor under cross-lingual conditions, emerging as both the strongest interference source and the most frequent fallback language. To mitigate this, we propose Soft Constrained Decoding (SCD), a lightweight, training-free decoding strategy that gently steers generation toward the target language by penalizing non-target-language tokens. SCD is model-agnostic and can be applied to any generation algorithm without modifying the architecture or requiring additional data. Experiments across three multilingual datasets and multiple typologically diverse languages show that SCD consistently improves language alignment and task performance, providing an effective and generalizable solution in multilingual RAG.

多语言RAG语言漂移解码优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。