用类脑学习规则让神经网络更省电，分类更高效
Energy-Efficient Information Representation in MNIST Classification Using Biologically Inspired Learning
- 模仿大脑结构可塑性优化突触使用，避免过度参数化
- 在手写数字识别任务中，比反向传播更省能且存得更多
- 适合想做节能、可扩展智能系统的研究者参考
高效表征学习对信息存储与分类至关重要，但常被人工神经网络忽视。这导致网络参数量可能高达13倍冗余，增加冗余和能耗。随着大语言模型规模扩大，此类问题愈发突出，引发重大伦理与环境担忧。本文基于此前提出的类脑学习规则，结合信息论分析其在MNIST分类任务中的效率表现。该规则模拟大脑结构可塑性，通过优化突触使用，自动保留必要数量的连接，有效防止过参数化。实验表明，该方法在能效和存储容量上优于反向传播（BP），无需预设网络架构，提升适应性，并体现大脑为新记忆预留空间的能力。此方法推动了可扩展、节能型人工智能发展，为构建资源高效、自适应的类脑模型提供可行框架。
原文摘要 · Abstract (English)
Efficient representation learning is essential for optimal information storage and classification. However, it is frequently overlooked in artificial neural networks (ANNs). This neglect results in networks that can become overparameterized by factors of up to 13, increasing redundancy and energy consumption. As the demand for large language models (LLMs) and their scale increase, these issues are further highlighted, raising significant ethical and environmental concerns. We analyze our previously developed biologically inspired learning rule using information-theoretic concepts, evaluating its efficiency on the MNIST classification task. The proposed rule, which emulates the brain's structural plasticity, naturally prevents overparameterization by optimizing synaptic usage and retaining only the essential number of synapses. Furthermore, it outperforms backpropagation (BP) in terms of efficiency and storage capacity. It also eliminates the need for pre-optimization of network architecture, enhances adaptability, and reflects the brain's ability to reserve 'space' for new memories. This approach advances scalable and energy-efficient AI and provides a promising framework for developing brain-inspired models that optimize resource allocation and adaptability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。