提出SAL方法，无需反向传播即可实现高效深度网络训练
SAL: Selective Adaptive Learning for Backpropagation-Free Training with Sparsification
- 通过选择性激活参数与自适应分区，解耦不同语义模式的梯度
- 在10个基准上表现优异，128层深度模型仍保持稳定收敛
- 受生物学习启发，适合研究新型神经网络训练范式
标准深度学习依赖反向传播（BP），其受制于生物上不合理的权重对称性，并在密集表示中面临显著梯度干扰。为此，我们提出选择性自适应学习（SAL），结合选择性参数激活与自适应区域划分。具体而言，SAL将参数空间分解为相互排斥、依赖样本的区域，从而缓解不同语义模式间的梯度干扰，并通过改进的反馈对齐消除显式的权重对称要求。实验表明，SAL在10个标准基准上展现出具有竞争力的收敛速度，提升了分类性能。此外，即使在深度高达128层、模型规模达10亿参数的情况下，SAL仍保持数值一致性与竞争性准确率。该方法受生物学习机制启发，为可扩展神经网络训练提供了更合理的替代方案。
原文摘要 · Abstract (English)
Standard deep learning relies on Backpropagation (BP), which is constrained by biologically implausible weight symmetry and suffers from significant gradient interference within dense representations. To mitigate these bottlenecks, we propose Selective Adaptive Learning (SAL), a training method that combines selective parameter activation with adaptive area partitioning. Specifically, SAL decomposes the parameter space into mutually exclusive, sample-dependent regions. This decoupling mitigates gradient interference across divergent semantic patterns and addresses explicit weight symmetry requirements through our refined feedback alignment. Empirically, SAL demonstrates competitive convergence rates, leading to improved classification performance across 10 standard benchmarks. Additionally, SAL achieves numerical consistency and competitive accuracy even in deep regimes (up to 128 layers) and large-scale models (up to 1B parameters). Our approach is loosely inspired by biological learning mechanisms, offering a plausible alternative that contributes to the study of scalable neural network training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。