发现多语言推理的真正关键特征,挑战以英语为中心的设计。
What Makes Good Multilingual Reasoning? Disentangling Reasoning Traces with Measurable Features
- 定义可量化的多语言推理特征,覆盖对齐、步骤与流程
- 多数特征与准确率正相关,但效果因语言而异甚至逆转
- 建议采用适配各语言特性的动态推理目标
大型推理模型在英语与其他语言之间仍存在显著性能差距,现有研究常假设只需让非英语推理模仿英语即可解决。本文质疑这一假设,提出:多语言有效推理的本质是什么?基于英语的推理特征在其他语言中是否真正有效?我们构建了一套涵盖多语言对齐、推理步骤与推理流程的可测量特征体系,通过逻辑回归量化各特征与最终答案准确率的关联性。进一步在多语言推理轨迹上训练稀疏自编码器,自动发现表征这些特征的潜在推理概念。最后,将特征作为推理时的选择策略,检验其能否引导模型实现更强的多语言推理。在两个数学推理基准、四种大型推理模型和十种语言上,我们发现大多数特征与准确率呈正相关,但相关强度在不同语言间差异显著,部分语言甚至出现反向关联。结果挑战了以英语为中心的奖励设计,指向需适应语言特异性的动态目标,为多语言评测与奖励机制设计提供具体启示。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) still exhibit large performance gaps between English and other languages, yet much current work assumes these gaps can be closed simply by making reasoning in every language resemble English reasoning. This work challenges this assumption by asking instead: what actually characterizes effective reasoning in multilingual settings, and to what extent do English-derived reasoning features genuinely help in other languages? We first define a suite of measurable reasoning features spanning multilingual alignment, reasoning step, and reasoning flow aspects of reasoning traces, and use logistic regression to quantify how each feature associates with final answer accuracy. We further train sparse autoencoders over multilingual traces to automatically discover latent reasoning concepts that instantiate or extend these features. Finally, we use the features as test-time selection policies to examine whether they can steer models toward stronger multilingual reasoning. Across two mathematical reasoning benchmarks, four LRMs, and 10 languages, we find that most features are positively associated with accuracy, but the strength of association varies considerably across languages and can even reverse in some. Our findings challenge English-centric reward designs and point toward adaptive objectives that accommodate language-specific reasoning patterns, with concrete implications for multilingual benchmark and reward design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。