让神经网络在宽度、深度和时间上自动适应资源约束，模拟生物智能。
Growing a Neural Network in Breadth, Depth, and Time
- 通过可微分代价函数联合优化网络宽度、深度与时间开销。
- 不同任务复杂度下，网络自适应扩展三维度，精度不变时可相互替代。
- 模型耗时与人类反应时间高度相关，适合研究脑科学与神经架构设计。
空间与时间资源约束对生物与人工智能系统至关重要。本文将循环卷积神经网络视为无限晶格的有限子集，定义了宽度、深度与时间的可微分代价项，并通过反向传播联合优化这些代价与任务误差。通过施加不同的资源压力，训练过程中自然涌现出多样化的计算图。我们发现三者可在保持特定精度的前提下相互权衡。随着任务复杂度提升，网络在三个维度上均自发增长；当输入被遮挡时，模型会自发增加递归步数。令人惊讶的是，模型所用时间与人类在物体识别任务中的反应时间高度相关。该框架为资源约束如何塑造神经结构提供了规范性解释，连接神经科学中关于大脑设计的问题，或有助于理解自然中多样的神经解决方案。
原文摘要 · Abstract (English)
Spatial and temporal resource constraints are critical for both biological and artificial intelligent systems. Here we define differentiable cost terms for breadth, depth, and time within a recurrent convolutional neural network conceived as a finite subset of an infinite lattice. We optimize these costs jointly with task errors via backpropagation. We set different pressures on breadth, depth, and time, which leads to diverse computational graphs emerging organically through training. We find that all three resources can be traded off against each other to achieve a given level of accuracy. Networks grow in all three dimensions with task complexity and spontaneously take more recurrent steps when inputs are occluded. Surprisingly, time used by the model correlates with human reaction times in an object recognition task. Our framework provides a normative account of how resource constraints shape neural architectures, connecting to questions about brain design in neuroscience, and may help illuminate the diversity of neural solutions found in nature.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。