arXiv:2412.05126cs.LGcs.NE2024-12被引 1

天然的神经元差异能让计算更高效,省电又稳定。

Skewed Neuronal Heterogeneity Enhances Efficiency On Various Computing Systems

  • 用生物内生的神经元时间常数差异替代人工优化
  • 在多种任务中提升性能与鲁棒性,且无需大模型
  • 适合低功耗神经形态芯片和真实生物系统设计

异质性是许多生物系统的基本特征,对计算具有深远影响。尽管可针对特定任务优化神经元与突触异质性,但此类自上而下的优化在生物学上不成立,易导致灾难性遗忘,并且数据与能耗极高。相比之下,生物体以极低代谢成本完成多种任务,其异质性为内在、成年后稳定且任务无关。受此启发,我们研究了神经元时间常数的自然变异在解决数百种不同复杂度的时序任务中的作用。结果表明,内在异质性以实现无关方式显著提升性能与鲁棒性,适用于(率编码)机器学习与(脉冲编码)神经形态应用。重要的是,仅偏斜的异质性分布——类似生物体中所见——能带来性能提升。进一步证明,这种计算优势使小规模网络即可达到良好表现,从而在模拟、生物体内及神经形态硬件上分别大幅降低运行、代谢与能量消耗。最后,讨论了内在异质性(而非任务诱导)对高效人工系统设计的意义,尤其适用于具类似器件间变异的新一代神经形态设备。

原文摘要 · Abstract (English)

Heterogeneity is a ubiquitous property of many biological systems and has profound implications for computation. While it is conceivable to optimize neuronal and synaptic heterogeneity for a specific task, such top-down optimization is biologically implausible, prone to catastrophic forgetting, and both data- and energy-intensive. In contrast, biological organisms, with remarkable capacity to perform numerous tasks with minimal metabolic cost, exhibit a heterogeneity that is inherent, stable during adulthood, and task-unspecific. Inspired by this intrinsic form of heterogeneity, we investigate the utility of variations in neuronal time constants for solving hundreds of distinct temporal tasks of varying complexity. Our results show that intrinsic heterogeneity significantly enhances performance and robustness in an implementation-independent manner, indicating its usefulness for both (rate-based) machine learning and (spike-coded) neuromorphic applications. Importantly, only skewed heterogeneity profiles-reminiscent of those found in biology-produce such performance gains. We further demonstrate that this computational advantage eliminates the need for large networks, allowing comparable performance with substantially lower operational, metabolic, and energetic costs, respectively in silico, in vivo, and on neuromorphic hardware. Finally, we discuss the implications of intrinsic (rather than task-induced) heterogeneity for the design of efficient artificial systems, particularly novel neuromorphic devices that exhibit similar device-to-device variability.

神经形态异质性低功耗计算效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。