用递归扩散生成上亿参数模型，单卡完成且性能接近训练好的网络。
Recurrent Diffusion for Large-Scale Parameter Generation
- 将参数分块为非重叠令牌，通过递归机制学习块间关系。
- 在ImageNet、COCO等任务上生成的参数性能媲美完整训练模型。
- 可生成未见过任务的参数，适合动态开放场景下的模型自生成。
参数生成长期难以匹配当前大规模视觉与语言模型的规模，限制了其广泛应用。本文提出针对大规模参数生成的递归扩散框架（RPG），可在单张GPU上生成达数亿参数的完整神经网络。方法首先将网络参数划分为互不重叠的令牌，每个对应模型的一部分；再通过递归机制学习令牌间的关联，生成原型作为扩散过程的条件，最终合成完整参数。在ResNets、ConvNeXts、ViTs于ImageNet 1K和COCO上的多种架构与任务中，RPG性能与全量训练模型相当，同时避免了过度内存开销。尤为关键的是，它能泛化至训练集外的未见任务，生成有效参数，展现出在动态、开放场景中的高度灵活性。RPG突破了长期存在的内存与可扩展性瓶颈，是人工智能生成人工智能的重要进展，有望实现此前被认为不可行的大规模高效权重生成。
原文摘要 · Abstract (English)
Parameter generation has long struggled to match the scale of today large vision and language models, curbing its broader utility. In this paper, we introduce Recurrent Diffusion for Large Scale Parameter Generation (RPG), a novel framework that generates full neural network parameters up to hundreds of millions on a single GPU. Our approach first partitions a networks parameters into non-overlapping tokens, each corresponding to a distinct portion of the model. A recurrent mechanism then learns the inter token relationships, producing prototypes which serve as conditions for a diffusion process that ultimately synthesizes the full parameters. Across a spectrum of architectures and tasks including ResNets, ConvNeXts and ViTs on ImageNet 1K and COCO, and even LoRA based LLMs RPG achieves performance on par with fully trained networks while avoiding excessive memory overhead. Notably, it generalizes beyond its training set to generate valid parameters for previously unseen tasks, highlighting its flexibility in dynamic and open ended scenarios. By overcoming the longstanding memory and scalability barriers, RPG serves as a critical advance in AI generating AI, potentially enabling efficient weight generation at scales previously deemed infeasible.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。