让大模型逐步中英混用推理，提升非英语语言表现
Gradual Code-Switching as Inference-Time Cross-Lingual Representational Alignment for LLMs
- 推理时渐进式中英混用，对齐非英语输入与英语思维空间
- 跨语言平均提升6.0个百分点，低资源语言提升达14.7个百分点
- 适合需要公平多语言支持的场景，如低资源语言应用
尽管大语言模型在多语言场景中取得进展，但其表现仍存在语言差异，主要因模型依赖以英语为中心的潜在表征。本文提出推理时代码切换（CSICL），一种跨语言表征对齐机制。不同于突兀的翻译转换，CSICL通过逐步从目标语言过渡到英语，引导推理路径，使非英语输入与英语主导的推理空间对齐。在4个大模型、6个数据集和10种语言上，CSICL consistently优于跨语言上下文学习基线，在目标语言和未见语言上分别获得6.0和4.8个百分点的平均提升。该效果在不同语系间泛化良好，尤其在低资源环境下更显著，目标语言提升14.7个百分点，未见语言提升5.3个百分点。结果表明，代码切换是推理阶段缓解跨语言错位的有效方法，推动大模型向更均衡高效的多语言系统演进。
原文摘要 · Abstract (English)
While large language models (LLMs) have achieved notable progress in multilingual settings, their performance remains uneven across languages as LLMs often rely on English-centric latent representations. In this work, we introduce code-switching in-context learning (CSICL), an inference-time mechanism for cross-lingual representational alignment. Rather than relying on an abrupt translation pivot, CSICL explicitly scaffolds the reasoning trajectory by gradually transitioning from a target language to English, aligning non-English inputs with an English-centric reasoning space. Across 4 LLMs, 6 datasets, and 10 languages, CSICL consistently outperforms cross-lingual in-context learning baselines, yielding average gains of 6.0pp and 4.8pp in target and unseen languages, respectively. The improvements generalize across language families and are even more pronounced in low-resource settings, with gains of 14.7pp in target and 5.3pp in unseen languages. These findings establish code-switching as a robust and effective approach for reducing cross-lingual misalignment during inference, moving LLMs towards more equitable and effective multilingual systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。