8B模型后训练新方案,公开全流程数据与方法,显著提升推理与指令遵循能力。
NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research

- 构建全透明数据流水线,包含384万样本和20万可验证候选集。
- 指令分支使基准分从55.01提升至61.85,推理分支达75.17分。
- 仅用4K样本即超越强化学习基线,适合轻量级模型优化研究。
后训练对齐决定大语言模型的推理与人类偏好遵循能力,但现有工作常隐瞒数据构建、过滤规则与训练细节,阻碍复现与轻量模型优化。本文提出NebulaExp,基于Qwen3-8B-base的全透明、消融驱动后训练流程,涵盖通用指令与复杂推理两类模型分支。构建384万条多源SFT样本原始语料与20万条可验证的RL候选池,设计端到端数据处理栈,包括响应蒸馏、多维交叉验证过滤、细粒度难度分级、任务分类与多样性采样。指令分支采用三阶段优化的监督微调,使平均基准分从Qwen3-8B-nothink的55.01提升至60.99;后续GRPO强化学习进一步增至61.85。推理分支中,中等难度GRPO RL将平均推理得分从73.88提升至75.17。为缓解强化学习对验证器的依赖,系统研究单教师与多教师OPD(MOPD):仅用4K指令遵循样本即在IFEval上超越基线3.26分,平均整体提升4.43;MOPD融合四名领域专家教师,仅需10K样本,较基线平均提升4.18。本报告提供8B规模模型可复现的后训练完整方案,并全面剖析指令遵循、数学推理、代码生成与通用知识间的权衡关系。
原文摘要 · Abstract (English)
Post-training alignment determines the reasoning and human preference following capabilities of large language models, yet most existing works withhold detailed data construction, filtering rules and training recipes, which hinders community reproducibility and lightweight model optimization. This work presents NebulaExp, a fully transparent, ablation-driven post-training pipeline built on Qwen3-8B-base, covering two orthogonal model branches: general instruct model and complex reasoning-specialized model. We curate a raw corpus of 3.84M multi-source SFT samples and a 200K verifiable RL candidate pool, and design an end-to-end data processing stack including response distillation, multi-dimensional cross-verification filtering, fine-grained difficulty grading, task classification and diversity-aware sampling. For the Instruct branch, our three-stage optimized supervised fine-tuning approach NebulaExp-Ins-SFT improves the average benchmark score from the 55.01 baseline of Qwen3-8B-nothink to 60.99. GRPO reinforcement learning then further elevates the average score to 61.85. For the Reasoning branch, medium-difficulty GRPO RL improves average reasoning score from 73.88 to 75.17. To address RL's dependency on task verifiers, we systematically investigate single-teacher and multi-teacher OPD (MOPD): utilizing merely 4K instruction-following samples and outperforms RL baseline by 3.26 points on IFEval with +4.43 average overall gain; MOPD fuses four domain-specialist teachers with merely 10K samples, lifting average performance by 4.18 over the base model. This report provides a fully reproducible empirical post-training recipe for 8B-scale LLMs, and comprehensively dissects the capability trade-offs among instruction adherence, mathematical reasoning, code generation and general knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。