arXiv:2410.06232q-bio.NCcs.AI2024-10ICLR被引 9

数据范围分布决定神经表征是否模块化,而非独立性。

Range, not Independence, Drives Modularity in Biologically Inspired Representations

论文配图:Range, not Independence, Drives Modularity in Biologically Inspired Representations
图 1 · 摘自论文原文
  • 基于非负性与能效的生物启发模型,提出模块化条件
  • 数据支持范围越广,越易形成单变量模块化表征
  • 适用于神经科学与人工网络,解释混合选择性新机制

为何生物与人工神经元有时仅编码单一有意义变量(模块化),有时却混合多变量?本文建立了一套理论,解释在非负且能效高的生物启发网络中,源变量表征何时会模块化。我们推导出线性自编码器中实现模块化的充分必要条件,其取决于源数据样本的特性。该理论适用于任意数据集,不再局限于以往研究中的统计独立假设。结果表明,当源变量的支持集“足够分散”时,系统会模块化。基于此理论,我们在多种非线性前馈与递归网络上验证了数据分布对模块化的影响,涵盖监督与无监督任务。进一步应用于海马旁皮层记录数据,发现范围独立性可解释空间与奖励信息在不同实验中的混杂或分离现象。此外,本研究提出混合选择性的新起源,超越了现有以灵活非线性分类为主的主流理论。总体而言,该理论为大脑与机器中的模块化表征提供了精确预测与设计工具。

原文摘要 · Abstract (English)

Why do biological and artificial neurons sometimes modularise, each encoding a single meaningful variable, and sometimes entangle their representation of many variables? In this work, we develop a theory of when biologically inspired networks -- those that are nonnegative and energy efficient -- modularise their representation of source variables (sources). We derive necessary and sufficient conditions on a sample of sources that determine whether the neurons in an optimal biologically-inspired linear autoencoder modularise. Our theory applies to any dataset, extending far beyond the case of statistical independence studied in previous work. Rather we show that sources modularise if their support is ``sufficiently spread''. From this theory, we extract and validate predictions in a variety of empirical studies on how data distribution affects modularisation in nonlinear feedforward and recurrent neural networks trained on supervised and unsupervised tasks. Furthermore, we apply these ideas to neuroscience data, showing that range independence can be used to understand the mixing or modularising of spatial and reward information in entorhinal recordings in seemingly conflicting experiments. Further, we use these results to suggest alternate origins of mixed-selectivity, beyond the predominant theory of flexible nonlinear classification. In sum, our theory prescribes precise conditions on when neural activities modularise, providing tools for inducing and elucidating modular representations in brains and machines.

神经表征模块化数据分布生物启发

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。