一个模型内嵌多个大小子模型,零样本提取,大幅降低推理模型训练成本。
Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs
- 通过嵌套子模型共享权重,单模型支持多尺寸部署。
- 仅用1100亿tokens生成9B和6B模型,成本降360倍。
- 适合需要多版本推理模型的工程部署场景。
训练针对多种规模和部署目标的大型语言模型代价高昂,需为每种尺寸单独训练。尽管剪枝与知识蒸馏等压缩技术降低了成本,但仍需数百亿token的训练量。本文提出Nemotron Elastic框架,构建包含嵌套子模型的推理型LLM,支持不同部署配置与预算。各子模型可零样本从父模型中提取,无需额外训练。通过端到端训练的路由器与专为推理模型设计的两阶段训练流程实现该功能。引入分组感知的SSM弹性化、异构MLP弹性化、基于归一化MSE的层重要性评估,以及支持多预算同时优化的知识蒸馏。在Nemotron Nano V2 12B上,仅用1100亿训练令牌即生成9B与6B模型,相较从头训练成本降低超360倍,相比当前最优压缩技术也降低约7倍。各嵌套模型性能达或优于现有最佳水平。且该方法支持‘一模型多版本’,部署内存恒定,不随模型家族数量增加。
原文摘要 · Abstract (English)
Training a family of large language models targeting multiple scales and deployment objectives is prohibitively expensive, requiring separate training runs for each different size. Recent work on model compression through pruning and knowledge distillation has reduced this cost; however, this process still incurs hundreds of billions of tokens worth of training cost per compressed model. In this paper, we present Nemotron Elastic, a framework for building reasoning-oriented LLMs, including hybrid Mamba-Attention architectures, that embed multiple nested submodels within a single parent model, each optimized for different deployment configurations and budgets. Each of these submodels shares weights with the parent model and can be extracted zero-shot during deployment without additional training or fine-tuning. We enable this functionality through an end-to-end trained router, tightly coupled to a two-stage training curriculum designed specifically for reasoning models. We additionally introduce group-aware SSM elastification that preserves Mamba's structural constraints, heterogeneous MLP elastification, normalized MSE-based layer importance for improved depth selection, and knowledge distillation enabling simultaneous multi-budget optimization. We apply Nemotron Elastic to the Nemotron Nano V2 12B model, simultaneously producing a 9B and a 6B model using only 110B training tokens; this results in over 360x cost reduction compared to training model families from scratch, and around 7x compared to SoTA compression techniques. Each of the nested models performs on par or better than the SoTA in accuracy. Moreover, unlike other compression methods, the nested capability of our approach allows having a many-in-one reasoning model that has constant deployment memory against the number of models in the family.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。