将神经网络中的结构与数值知识分离，训练速度提升40%。
GLAI: GreenLightningAI for Accelerated Training through Knowledge Decoupling
- 分离激活模式（结构）与权重数值（量），只优化后者以加速训练。
- 在多种任务中，精度相当但训练时间平均减少40%。
- 可替换各类MLP模块，适合需要高效训练的模型设计。
本文提出GreenLightningAI（GLAI），一种替代传统MLP的新型架构块。核心思想是将训练中常被混淆的两类知识解耦：(i) 由ReLU激活诱导的稳定激活模式所编码的结构知识；(ii) 由数值权重和偏置携带的量化知识。在结构稳定后固定不变，GLAI将MLP重构成多路径组合，仅优化量化部分。该设计保留了MLP的通用逼近能力，同时显著提升训练效率，在本研究考察的各类场景中平均训练时间缩短约40%。关键的是，GLAI不仅是分类器，更是一种通用模块，可替代从冻结主干的监督头到自监督学习中的投影层或少样本分类器中的任意MLP。在多样实验设置中，其精度与等参数量的MLP相当甚至更优，且收敛更快。总体而言，GLAI确立了一种新设计范式，为未来在以MLP为主导计算开销的大型架构（如Transformer）中集成提供了可能。
原文摘要 · Abstract (English)
In this work we introduce GreenLightningAI (GLAI), a new architectural block designed as an alternative to conventional MLPs. The central idea is to separate two types of knowledge that are usually entangled during training: (i) *structural knowledge*, encoded by the stable activation patterns induced by ReLU activations; and (ii) *quantitative knowledge*, carried by the numerical weights and biases. By fixing the structure once stabilized, GLAI reformulates the MLP as a combination of paths, where only the quantitative component is optimized. This reformulation retains the universal approximation capabilities of MLPs, yet achieves a more efficient training process, reducing training time by ~40% on average across the cases examined in this study. Crucially, GLAI is not just another classifier, but a generic block that can replace MLPs wherever they are used, from supervised heads with frozen backbones to projection layers in self-supervised learning or few-shot classifiers. Across diverse experimental setups, GLAI consistently matches or exceeds the accuracy of MLPs with an equivalent number of parameters, while converging faster. Overall, GLAI establishes a new design principle that opens a direction for future integration into large-scale architectures such as Transformers, where MLP blocks dominate the computational footprint.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。