用生物启发的符号对称规则微调预训练模型，提升鲁棒性。
Sign-Symmetry Learning Rules are Robust Fine-Tuners
- 用符号对称规则替代反向传播进行模型微调
- 在多个任务上保持与反向传播相当的性能
- 适合追求模型鲁棒性和生物合理性研究者
反向传播（BP）长期以来是训练神经网络的主要方法，因其高效性而被广泛采用。然而，许多替代方法——统称为反馈对齐——被提出以寻找更符合生物学机制的学习方式。尽管这些方法具有理论吸引力，但其性能普遍低于BP，导致研究兴趣下降。本文重新审视这些方法的作用，探索其如何融入标准神经网络训练流程。具体而言，我们提出使用符号对称学习规则对预训练的BP模型进行微调，并证明该方法不仅维持了与BP相当的性能，还提升了模型鲁棒性。通过在多个任务和基准上的大量实验，验证了该方法的有效性。研究结果为神经网络训练提供了新视角，并开辟了利用生物启发学习规则开展深度学习研究的新方向。
原文摘要 · Abstract (English)
Backpropagation (BP) has long been the predominant method for training neural networks due to its effectiveness. However, numerous alternative approaches, broadly categorized under feedback alignment, have been proposed, many of which are motivated by the search for biologically plausible learning mechanisms. Despite their theoretical appeal, these methods have consistently underperformed compared to BP, leading to a decline in research interest. In this work, we revisit the role of such methods and explore how they can be integrated into standard neural network training pipelines. Specifically, we propose fine-tuning BP-pre-trained models using Sign-Symmetry learning rules and demonstrate that this approach not only maintains performance parity with BP but also enhances robustness. Through extensive experiments across multiple tasks and benchmarks, we establish the validity of our approach. Our findings introduce a novel perspective on neural network training and open new research directions for leveraging biologically inspired learning rules in deep learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。