让神经网络的权重随输入动态调整,提升适应能力。
Bio-Inspired Adaptive Neurons for Dynamic Weighting in Artificial Neural Networks
- 用输入信号驱动权重变化,通过切比雪夫多项式实现动态调节。
- 145个数据集测试中,优于传统MLP的有121个,性能持平24个。
- 兼容现有架构,适合处理动态或非线性数据的任务。
传统神经网络在推理时采用固定权重,难以适应输入变化,而生物神经元能根据刺激动态调节信号强度。为弥合这一差距,我们提出一种新型自适应神经网络框架,将神经元权重建模为输入信号的函数,实现实时动态调整。该方法在保持传统人工神经网络结构的基础上引入自适应能力。研究中采用切比雪夫多项式作为分解方式,其系数通过训练学习。在145个数据集上测试,所提自适应切比雪夫神经网络在约8%的情况下显著优于等效MLP,在121个数据集上表现更优,其余24个数据集性能与MLP持平。该模型作为MLP的推广形式,既保留了标准神经网络的泛化能力,又在复杂任务中展现出更高准确率。结果表明,自适应神经元可增强网络的泛化性、灵活性和鲁棒性,尤其适用于具有动态或非线性依赖关系的应用场景。
原文摘要 · Abstract (English)
Traditional neural networks employ fixed weights during inference, limiting their ability to adapt to changing input conditions, unlike biological neurons that adjust signal strength dynamically based on stimuli. This discrepancy between artificial and biological neurons constrains neural network flexibility and adaptability. To bridge this gap, we propose a novel framework for adaptive neural networks, where neuron weights are modeled as functions of the input signal, allowing the network to adjust dynamically in real-time. Importantly, we achieve this within the same traditional architecture of an Artificial Neural Network, maintaining structural familiarity while introducing dynamic adaptability. In our research, we apply Chebyshev polynomials as one of the many possible decomposition methods to achieve this adaptive weighting mechanism, with polynomial coefficients learned during training. Out of the 145 datasets tested, our adaptive Chebyshev neural network demonstrated a marked improvement over an equivalent MLP in approximately 8\% of cases, performing strictly better on 121 datasets. In the remaining 24 datasets, the performance of our algorithm matched that of the MLP, highlighting its ability to generalize standard neural network behavior while offering enhanced adaptability. As a generalized form of the MLP, this model seamlessly retains MLP performance where needed while extending its capabilities to achieve superior accuracy across a wide range of complex tasks. These results underscore the potential of adaptive neurons to enhance generalization, flexibility, and robustness in neural networks, particularly in applications with dynamic or non-linear data dependencies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。