通过神经元调优，让大模型在韩语中更好自我纠错。
Do LLMs Need Inherent Reasoning Before Reinforcement Learning? A Study in Korean Self-Correction
- 针对韩语输入，调优早期层的特定神经元以对齐推理过程。
- 自纠正数据集使数学推理和纠错能力提升显著。
- 关键不是加新知识，而是激活已有推理能力。
大型语言模型在英语等高资源语言中表现出强大的推理与自纠正能力，但在韩语等低资源语言中表现有限。本研究探讨强化学习（RL）能否使韩语推理能力达到英语水平。结果表明，若模型本身缺乏内在韩语推理能力，仅靠强化学习提升有限。为此，我们探索多种微调策略，发现将模型内部推理过程与韩语输入对齐——尤其是调优早期层的韩语特异性神经元——是发挥强化学习效能的关键。我们构建了一个自纠正代码切换数据集以促进对齐，在数学推理和自纠正任务中均观察到显著性能提升。最终结论：多语言推理增强的关键并非注入新语言知识，而是有效激发并对齐已有的推理能力。本研究为内部翻译与神经元级调优如何促进多语言推理对齐提供了新视角。
原文摘要 · Abstract (English)
Large Language Models (LLMs) demonstrate strong reasoning and self-correction abilities in high-resource languages like English, but their performance remains limited in low-resource languages such as Korean. In this study, we investigate whether reinforcement learning (RL) can enhance Korean reasoning abilities to a degree comparable to English. Our findings reveal that RL alone yields limited improvements when applied to models lacking inherent Korean reasoning capabilities. To address this, we explore several fine-tuning strategies and show that aligning the model's internal reasoning processes with Korean inputs-particularly by tuning Korean-specific neurons in early layers-is key to unlocking RL's effectiveness. We introduce a self-correction code-switching dataset to facilitate this alignment and observe significant performance gains in both mathematical reasoning and self-correction tasks. Ultimately, we conclude that the crucial factor in multilingual reasoning enhancement is not injecting new linguistic knowledge, but effectively eliciting and aligning existing reasoning capabilities. Our study provides a new perspective on how internal translation and neuron-level tuning contribute to multilingual reasoning alignment in LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。