arXiv:2510.26336cs.CLcs.AI2025-10被引 2

用自动课程学习让大模型从通用变专精,不丢原有能力。

From Amateur to Master: Infusing Knowledge into LLMs via Automated Curriculum Learning

  • 自动生成按认知层级设计的问答课程,分阶段提升难度。
  • 在微经济学等难点领域提升5个百分点,整体平均增3点。
  • 防止遗忘还促进跨领域知识迁移,适合专业场景增强模型。

大型语言模型在通用任务上表现优异,但在经济学、心理学等需要深入理解的领域表现不足。为解决此问题,我们提出ACER(自动化课程增强训练方案),在不牺牲通用能力的前提下,将通用模型转化为领域专家。ACER首先通过生成学科目录并依据布卢姆分类法创建问答对,构建系统化、逐步升级的教材式课程。基于该合成语料,采用交错式课程调度进行持续预训练,同时对齐内容与认知维度的学习进程。在Llama 3.2(1B和3B)上的实验表明,在MMLU专项测试中取得显著提升:在微经济学等困难领域,准确率提高5个百分点;所有目标领域平均提升3个百分点。值得注意的是,ACER不仅避免灾难性遗忘,还带来正向跨域知识迁移,非目标领域性能提升0.7点。除MMLU外,对ARC和GPQA等知识密集型基准测试性能提升超2个百分点,且保持通用推理任务稳定。结果表明,ACER是一种可扩展、高效的弥补大模型领域差距的方法。

原文摘要 · Abstract (English)

Large Language Models (LLMs) excel at general tasks but underperform in specialized domains like economics and psychology, which require deep, principled understanding. To address this, we introduce ACER (Automated Curriculum-Enhanced Regimen) that transforms generalist models into domain experts without sacrificing their broad capabilities. ACER first synthesizes a comprehensive, textbook-style curriculum by generating a table of contents for a subject and then creating question-answer (QA) pairs guided by Bloom's taxonomy. This ensures systematic topic coverage and progressively increasing difficulty. The resulting synthetic corpus is used for continual pretraining with an interleaved curriculum schedule, aligning learning across both content and cognitive dimensions. Experiments with Llama 3.2 (1B and 3B) show significant gains in specialized MMLU subsets. In challenging domains like microeconomics, where baselines struggle, ACER boosts accuracy by 5 percentage points. Across all target domains, we observe a consistent macro-average improvement of 3 percentage points. Notably, ACER not only prevents catastrophic forgetting but also facilitates positive cross-domain knowledge transfer, improving performance on non-target domains by 0.7 points. Beyond MMLU, ACER enhances performance on knowledge-intensive benchmarks like ARC and GPQA by over 2 absolute points, while maintaining stable performance on general reasoning tasks. Our results demonstrate that ACER offers a scalable and effective recipe for closing critical domain gaps in LLMs.

课程学习领域专家大模型微调

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。