用基因瓶颈模拟生物神经发育，生成可高效处理时序任务的模块化网络。
Distilling a Modular Reservoir Through a Genomic Bottleneck
- 通过超网络学习基因式压缩生成过程，构建模块化递归网络。
- 仅需少量训练即可解决复杂时序任务，且保持强鲁棒性。
- 适合研究神经发育机制或高效时序模型的设计者。
生物神经网络的复杂结构主要在发育过程中由基因组中相对紧凑的蓝图引导形成。这一解码过程产生的连接具有丰富结构，使生物体出生时即具备功能模块。该初始结构作为骨架，可通过终身经验与多种可塑性机制逐步优化。受进化与发育学习交互的启发，我们使用超网络学习一种压缩的生成过程，以生成模块化储藏室的连接。结果表明，基于课程的元学习与模块化储藏室计算的结合，可生成稀疏的递归网络，在极少训练下即可解决困难的时序任务，且无需牺牲鲁棒性。
原文摘要 · Abstract (English)
The intricate structures of biological neural networks largely emerge during development, guided by a comparatively compressed blueprint encoded in the genome. The connectivity that emerges from this decoding process is rich in structure, and already equips the organism with functional modules upon birth. This initial structure serves as a scaffold that can be gradually refined and fine-tuned through lifelong experience, via a variety of plasticity mechanisms. Drawing inspiration from this interaction between evolutionary and developmental modes of learning, we use hypernetworks to learn a compressed generative process that generates the connectivity of a modular reservoir. We show that this marriage between curriculum-based meta-learning and modular reservoir computing can generate sparse recurrent networks that solve difficult temporal tasks with minimal training and without concessions to robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。