研究大模型中思考与非思考模式的融合训练，发现两者存在内在冲突。
Fusion Training for Mathematical Generalization in Large Language Models
- 通过调节思考与非思考数据比例和训练时序，系统分析融合训练机制。
- 增加非思考数据会显著降低思考模式的数学解题准确率。
- 提出最优训练策略依赖数据比例，适合大模型训练优化研究者参考。
思维模式融合（TMF）使大语言模型能够同时支持简洁回答和长篇推理，通过在单个模型中统一非思考模式与思考模式实现。然而，其训练动态，包括两种模式之间的数据比例和训练时序，仍缺乏深入探索。本文针对数学问题求解任务，构建了包含多种思考-非思考数据比例及三种训练时序的基准测试。结果表明，两种模式间存在不对称交互：提高非思考监督比例会显著降低思考模式的准确率。进一步发现，不同训练时序可调节这一权衡关系，且最优时序取决于数据比例。最后，量化出非思考与思考模式监督之间存在负相关性，揭示了二者间的固有张力。这些发现为设计有效的TMF训练设置提供了实用指导。所有代码与数据已公开于：https://github.com/caocongfeng/Fusion-Bench.git。
原文摘要 · Abstract (English)
Thinking Mode Fusion (TMF) enables large language models to support both concise responses and long-form reasoning by unifying a non-thinking mode and a thinking mode within a single model. However, its training dynamics, including the \emph{data ratio} and \emph{training schedule} between the two modes, remain underexplored. In this work, we present a systematic study of TMF by analyzing the effects of the training schedule and data ratio between thinking and non-thinking modes. Focusing on mathematical problem solving, we construct a benchmark with multiple thinking-to-non-thinking data ratios and three training schedules. Our results reveal an asymmetric interaction between the two modes: increasing the ratio of non-thinking supervision reduces the accuracy of the thinking mode. We further show that different training schedules modulate this trade-off and that the optimal schedule depends on the data ratio. Finally, we quantify a negative correlation between non-thinking and thinking mode supervision, highlighting an inherent tension between these two modes. These findings provide practical guidance for designing effective TMF training settings. All code and data are released to support further research at: \href{https://github.com/caocongfeng/Fusion-Bench.git}{\textbf{Fusion Bench}}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。