让大模型在相同训练损失下提升下游泛化能力,靠的是优化器引导更接近的极小值点。
Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima

- 设计Nexus优化器,通过最大化梯度相似性促进不同任务极小值点靠近
- 在30亿参数模型上,跨分布损失降低0.012,复杂推理任务准确率提升15%
- 适用于追求更强泛化能力的模型训练,尤其适合多源异构数据场景
大语言模型的基础能力源于在互联网规模、高度异质的数据混合上进行预训练。本文研究预训练收敛状态的几何特性:模型是否在所有数据源上收敛到同一极小值点(如图示中靠近的情况),还是仅达到总损失的极小值(如图示中相距较远的情况)?我们假设任务特定极小值之间的几何接近性与下游泛化能力密切相关。实验发现,标准优化器(如AdamW)常收敛于任务极小值彼此远离的点。为此,提出Nexus优化器,通过优化过程中最大化梯度相似性来促进极小值间的接近。在1.3亿至30亿参数模型、多种数据混合及超参调度下,实验表明Nexus显著提升下游性能,尽管预训练损失相同(见图示)。值得注意的是,在30亿模型上,跨分布损失减少0.012,复杂推理任务(如GSM8k)准确率最高提升15.0%。该发现挑战了以预训练损失作为模型评估唯一指标的惯常做法,揭示了隐式偏差对解锁下游泛化的重要性。
原文摘要 · Abstract (English)
The foundational capabilities of large language models are acquired during pretraining on internet-scale, highly heterogeneous data mixtures. In this work, we investigate an interesting geometric question regarding the converged state of pretraining: Does the model converge to a common minimizer across all data sources (e.g., \cref{fig:cwa_illustration:close}), or merely a minimizer of the summed loss (e.g., \cref{fig:cwa_illustration:distant})? We hypothesize that the geometric "closeness" of task-specific minima is intrinsically linked to downstream generalization. We reveal that standard optimizers (e.g., AdamW) often converge to points where task-specific minima are distant from each other. To address this, we propose the Nexus optimizer, which encourages the closeness of these minima by maximizing gradient similarity during optimization. Experiments across models ranging from 130M to 3B parameters, various data mixtures and hyperparameter schedules, show that Nexus \textit{significantly boosts downstream performance}, despite \textit{achieving the same pretraining loss} (see \cref{fig:demo:benchmark}). Notably, on the 3B model, Nexus reduces the out-of-distribution loss by 0.012 and yields up to a 15.0\% accuracy improvement on complex reasoning tasks (e.g., GSM8k). This finding challenges the reliance on pretraining loss as the sole proxy for model evaluation and demonstrates the importance of implicit biases in unlocking downstream generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。