抑制性归一化可提升神经网络学习效率，关键在于同时归一化误差信号。
Inhibitory normalization of error signals improves learning in neural circuits
- 在神经网络中引入兴奋与抑制神经元分离结构，模拟生物神经回路的抑制性归一化机制。
- 仅在推理时归一化输入无法提升性能，但若同步归一化反向传播的误差信号，准确率显著提高。
- 该研究为理解大脑如何通过抑制性神经元优化学习提供了新思路，适合神经科学与类脑计算研究者。
归一化是神经回路中的关键运算。在大脑中，已有证据表明归一化通过抑制性中间神经元实现，使神经种群能适应输入分布的变化。在人工神经网络（ANNs）中，归一化被用于改善复杂输入分布任务的学习表现。然而，生物神经回路中由抑制介导的归一化是否也能提升学习尚不明确。本文通过构建包含独立兴奋与抑制神经元群体的人工神经网络，在具有可变亮度的图像识别任务上进行训练，发现若仅在推理阶段应用归一化，学习性能并未提升；但当将归一化扩展至反向传播的误差信号时，性能显著改善。结果表明，若抑制性归一化在大脑中促进学习，则必须同时对学习信号进行归一化。
原文摘要 · Abstract (English)
Normalization is a critical operation in neural circuits. In the brain, there is evidence that normalization is implemented via inhibitory interneurons and allows neural populations to adjust to changes in the distribution of their inputs. In artificial neural networks (ANNs), normalization is used to improve learning in tasks that involve complex input distributions. However, it is unclear whether inhibition-mediated normalization in biological neural circuits also improves learning. Here, we explore this possibility using ANNs with separate excitatory and inhibitory populations trained on an image recognition task with variable luminosity. We find that inhibition-mediated normalization does not improve learning if normalization is applied only during inference. However, when this normalization is extended to include back-propagated errors, performance improves significantly. These results suggest that if inhibition-mediated normalization improves learning in the brain, it additionally requires the normalization of learning signals.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。