用振荡神经元模拟动态绑定，提升模型泛化与鲁棒性。
Artificial Kuramoto Oscillatory Neurons
- 基于柯朗莫托同步机制设计振荡神经元，实现动态绑定。
- 在无监督物体发现、对抗鲁棒性等任务上均显著提效。
- 适合关注神经动力学与动态表征的模型研究者。
长期以来，神经科学与人工智能领域均发现，神经元间的'绑定'机制会引发竞争性学习，使表征压缩以在深层网络中表达更抽象的概念。近期还提出，时空动态表征在神经科学与人工智能中扮演关键角色。本文引入人工柯朗莫托振荡神经元（AKOrN），作为阈值单元的动态替代方案，可兼容全连接、卷积或注意力等多种连接结构。其广义柯朗莫托更新通过同步动力学实现神经元间的绑定。实验表明，该方法在无监督物体发现、对抗鲁棒性、校准不确定性量化及推理等多项任务中均有性能提升。这些结果表明，重新思考神经表征最基本的神经元假设至关重要，尤其凸显了动态表征的重要性。
原文摘要 · Abstract (English)
It has long been known in both neuroscience and AI that ``binding'' between neurons leads to a form of competitive learning where representations are compressed in order to represent more abstract concepts in deeper layers of the network. More recently, it was also hypothesized that dynamic (spatiotemporal) representations play an important role in both neuroscience and AI. Building on these ideas, we introduce Artificial Kuramoto Oscillatory Neurons (AKOrN) as a dynamical alternative to threshold units, which can be combined with arbitrary connectivity designs such as fully connected, convolutional, or attentive mechanisms. Our generalized Kuramoto updates bind neurons together through their synchronization dynamics. We show that this idea provides performance improvements across a wide spectrum of tasks such as unsupervised object discovery, adversarial robustness, calibrated uncertainty quantification, and reasoning. We believe that these empirical results show the importance of rethinking our assumptions at the most basic neuronal level of neural representation, and in particular show the importance of dynamical representations. Code:https://github.com/autonomousvision/akorn Project page:https://takerum.github.io/akorn_project_page/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。