arXiv:2501.16154cs.CLcs.AI2025-01AAAI被引 13

通过动态选择思考语言,提升低资源语言的跨语言事实推理能力。

AdaMCoT: Rethinking Cross-Lingual Factual Reasoning through Adaptive Multilingual Chain-of-Thought

  • 用可变思考语言动态路由推理路径,不需额外预训练。
  • 在多语言基准上显著提升事实推理准确率与跨语言一致性。
  • 特别适合低资源语言场景,保持文化语义细节。

大型语言模型(LLMs)通过在多样化语料上预训练展现出强大的多语言能力。尽管模型具备较强推理能力,但因训练数据分布不均,不同语言间性能差异显著。现有方法依赖样本级翻译进行大规模多语言预训练和跨语言微调,存在扩展性差且难以捕捉跨语言细微推理过程的问题。本文提出AdaMCOT(自适应多语言思维链),通过在中间‘思考语言’中动态路由推理过程,再生成目标语言回答,提升多语言事实推理能力。该框架采用语言无关的核心结构,并引入基于奖励的自适应机制,无需额外预训练即可选择最优推理路径。在多个基准上的综合评估表明,该方法在事实推理质量与跨语言一致性方面均有显著提升,尤其在低资源语言设置下表现突出。对模型隐藏状态与语义空间的深入分析揭示了方法的有效机制。结果表明,自适应推理路径能有效缩小高、低资源语言间的性能差距,同时保留文化和语言细微差异。

原文摘要 · Abstract (English)

Large language models (LLMs) have shown impressive multilingual capabilities through pretraining on diverse corpora. Although these models show strong reasoning abilities, their performance varies significantly between languages due to the imbalanced distribution of training data. Existing approaches using sample-level translation for extensive multilingual pretraining and cross-lingual tuning face scalability challenges and often fail to capture nuanced reasoning processes across languages. In this paper, we introduce AdaMCOT (Adaptive Multilingual Chain-of-Thought), a framework that enhances multilingual factual reasoning by dynamically routing thought processes in intermediary "thinking languages" before generating target-language responses. AdaMCOT leverages a language-agnostic core and incorporates an adaptive, reward-based mechanism for selecting optimal reasoning pathways without requiring additional pretraining. Our comprehensive evaluation across multiple benchmarks demonstrates substantial improvements in both factual reasoning quality and cross-lingual consistency, with particularly strong performance gains in low-resource language settings. An in-depth analysis of the model's hidden states and semantic space further elucidates the underlying mechanism of our method. The results suggest that adaptive reasoning paths can effectively bridge the performance gap between high and low-resource languages while maintaining cultural and linguistic nuances.

多语言推理链低资源语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。