通过双损失优化,让大模型在专精领域提升能力的同时不丢通用技能。
MoL for LLMs: Dual-Loss Optimization to Enhance Domain Expertise While Preserving General Capabilities
- 用交叉熵和KL散度分别训练领域与通用语料,分离优化目标。
- 1:1语料比例下,数学推理任务准确率提升27.9%,难题集提升83.3%。
- 适合需要保持通用能力又想强化特定领域表现的模型应用。
尽管大语言模型在通用任务中表现良好,但在特定领域应用中仍面临幻觉和准确率限制。持续预训练(CPT)方法存在两大问题:(1) 领域偏倚数据会损害通用语言能力,(2) 语料混合比例不当限制有效适配。为此,我们提出一种新框架——损失混合(MoL),将领域与通用语料的优化目标解耦。具体而言,对领域语料使用交叉熵(CE)损失以确保知识获取,对通用语料则使用KL散度使训练与基座模型的基础能力对齐。该双损失架构在保留通用能力的同时增强领域专长,避免灾难性遗忘。实证表明,1:1领域-通用语料比例可最优平衡训练与过拟合,无需大量调参或高资源实验。相较于传统CPT方法常导致通用能力退化,本模型在Math-500基准上非思考模式下准确率提升27.9%,在挑战性的AIME25子集思考模式下提升83.3%,验证了方法有效性。
原文摘要 · Abstract (English)
Although large language models (LLMs) perform well in general tasks, domain-specific applications suffer from hallucinations and accuracy limitations. Continual Pre-Training (CPT) approaches encounter two key issues: (1) domain-biased data degrades general language skills, and (2) improper corpus-mixture ratios limit effective adaptation. To address these, we propose a novel framework, Mixture of Losses (MoL), which decouples optimization objectives for domain-specific and general corpora. Specifically, cross-entropy (CE) loss is applied to domain-corpus to ensure knowledge acquisition, while Kullback-Leibler (KL) divergence aligns general-corpus training with the base model's foundational capabilities. This dual-loss architecture preserves universal skills while enhancing domain expertise, avoiding catastrophic forgetting. Empirically, we validate that a 1:1 domain-to-general corpus ratio optimally balances training and overfitting without the need for extensive tuning or resource-intensive experiments. Furthermore, our experiments demonstrate significant performance gains compared to traditional CPT approaches, which often suffer from degradation in general language capabilities; our model achieves 27.9% higher accuracy on the Math-500 benchmark in the non-think reasoning mode, and an impressive 83.3% improvement on the challenging AIME25 subset in the think mode, underscoring the effectiveness of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。