arXiv:2604.15490cs.CL2026-04

教大模型在推理时智能切换语言,提升跨语言思维能力。

Think Multilingual, Not Harder: A Data-Efficient Framework for Teaching Reasoning Models to Code-Switch

论文配图:Think Multilingual, Not Harder: A Data-Efficient Framework for Teaching Reasoning Models to Code-Switch
图 1 · 摘自论文原文
  • 基于真实推理轨迹分析,设计可复用的多语言切换教学框架。
  • 仅用少量数据就显著提升模型产生有益语言混用的能力。
  • 适用于跨语言推理任务,对非语言混合任务也有效。

近期大型语言模型在数学、符号与逻辑等复杂任务上的推理能力不断提升。有趣的是,尽管这些模型通常训练为生成单一语言文本,但已观察到它们会自然地进行语言切换(即混用语言)。以往研究或视语言切换为错误,或通过调整输入提示或解码过程加以控制,或仅关注有限的语言、领域、任务与模型。本文首次提出一个语言学与行为动机驱动的微调框架,旨在识别并教授模型更有效的语言切换推理行为。首先,我们构建并系统分析了一个涵盖多种模型、语言、任务与领域的推理轨迹数据集,以理解现有模型中的语言切换类型。随后,基于对有益行为的观察,开发了针对性微调干预策略。实验表明,该框架能以极低的数据成本显著提升模型产生有益语言切换的能力。更令人意外的是,通过针对不直接涉及语言切换的任务(如机器翻译)进行微调,也能改变模型的语言切换行为。本工作表明,数据高效干预可有效引导模型掌握有帮助的跨语言推理模式。

原文摘要 · Abstract (English)

Recent developments in reasoning capabilities have enabled large language models to solve increasingly complex mathematical, symbolic, and logical tasks. Interestingly, while reasoning models are often trained to generate monolingual text, these models have also been observed to code-switch (i.e., mix languages). Prior works have either viewed code-switching as an undesirable error, attempted to control code-switching through modifications to input prompts or the output decoding process, or focus on narrow subsets of languages, domains, tasks, and models. We address these gaps by introducing the first linguistically and behaviorally motivated fine-tuning framework for identifying beneficial code-switched reasoning behaviors in large language models and teaching these models to code-switch more effectively for reasoning. First, we create and systematically analyze a dataset of reasoning traces from diverse models, languages, tasks, and domains to understand the types of code-switching behaviors found in existing reasoning models. Then, we develop fine-tuning interventions that teach reasoning models to code-switch based on our observations of helpful behaviors in existing models. We find that our framework can significantly increase beneficial code-switched reasoning behaviors in a data-efficient manner. Interestingly, we also find that code-switching behaviors in reasoning models can be modified by fine-tuning for tasks that do not directly demonstrate code-switching in reasoning (e.g., machine translation). Our work suggests that data-efficient interventions can instill helpful forms of code-switching behavior in reasoning models.

多语言推理语言切换模型微调数据效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。