用英语思维桥接濒危语言的推理能力,仅靠少量数据就实现突破。
Reasoning Transfer for an Extremely Low-Resource and Endangered Language: Bridging Languages Through Sample-Efficient Language Understanding
- 通过英语作为中间语言生成思维链,跨语言传递推理能力。
- 在爱尔兰语数学推理任务上提升最高达28.33%。
- 适合研究濒危语言、低资源多语言推理的学者使用。
尽管大型语言模型(LLMs)已能通过生成思维链(CoT)解决推理任务,但这些进展主要集中在高资源语言,对低资源语言帮助有限。本文在极端低资源场景下系统评估了提示、模型编辑和微调等方法。提出英语桥梁式思维链训练(English-Pivoted CoT Training),利用大模型内部表示与主流语言对齐的特性:输入目标语言时,先在英语中生成思维链,再输出目标语言的答案。在多个数学推理基准测试中,该方法相比基线最高提升28.33%。分析及混合语言思维链、两阶段训练等实验表明,显式分离语言理解与推理可增强跨语言推理能力。为推动后续研究,我们发布首个爱尔兰语数学任务基准数据集LC2024。结果与资源展示了一条无需为每种极低资源语言大量重训即可实现多语言推理的可行路径。
原文摘要 · Abstract (English)
Recent advances have enabled Large Language Models (LLMs) to tackle reasoning tasks by generating chain-of-thought (CoT) rationales, yet these gains have largely applied to high-resource languages, leaving low-resource languages behind. In this work, we first investigate CoT techniques in extremely low-resource scenarios through previous prompting, model-editing, and fine-tuning approaches. We introduce English-Pivoted CoT Training, leveraging the insight that LLMs internally operate in a latent space aligned toward the dominant language. Given input in a low-resource language, we perform supervised fine-tuning to generate CoT in English and output the final response in the target language. Across mathematical reasoning benchmarks, our approach outperforms other baselines with up to 28.33% improvement in low-resource scenarios. Our analysis and additional experiments, including Mixed-Language CoT and Two-Stage Training, show that explicitly separating language understanding from reasoning enhances cross-lingual reasoning abilities. To facilitate future work, we also release \emph{LC2024}, the first benchmark for mathematical tasks in Irish, an extremely low-resource and endangered language. Our results and resources highlight a practical pathway to multilingual reasoning without extensive retraining in every extremely low-resource language, despite data scarcity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。