无需反向传播，用前馈激活快速固化指令知识
QF: Quick Feedforward AI Model Training without Gradient Back Propagation
- 通过前馈激活直接更新权重，闭式求解避免梯度计算
- 仅需少量参数调整，且保留原有知识不遗忘
- 适合追求高效、类脑训练的AI系统研究者
我们提出快速前馈（QF）学习，一种针对基于Transformer模型的新颖知识固化框架，可通过前馈激活将指令知识高效注入模型权重，无需任何梯度反向传播。与传统微调不同，QF更新为闭式求解，仅需极少参数修改，并能保留先前知识。重要的是，QF允许模型在相同运行时环境中完成训练与推理，提升资源效率，更贴近人类大脑的工作方式。代码与模型已在GitHub开源。希望QF学习能启发更高效、类脑的AI系统范式。
原文摘要 · Abstract (English)
We propose Quick Feedforward (QF) Learning, a novel knowledge consolidation framework for transformer-based models that enables efficient transfer of instruction derived knowledge into model weights through feedforward activations without any gradient back propagation. Unlike traditional finetuning, QF updates are computed in closed form, require minimal parameter modification, and preserve prior knowledge. Importantly, QF allows models to train and infer within the same runtime environment, making the process more resource efficient and closely aligned with how the human brain operates. Code and models are open sourced on GitHub. I hope QF Learning inspires a more efficient and brain-like paradigm for AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。