用提前计算+并行过滤,让神经演化网格生成快10倍以上。
Tensor-Accelerated Eager Multi-Resolution Grids for Evolving Large-Scale Substrates
- 先全量计算所有分辨率位置,再按方差筛选,打破递归依赖
- 在深度5-7时实测GPU速度提升12-34倍,解题成功率更高
- 适合需要动态拓扑的大规模神经演化任务
在神经演化中,间接编码通过紧凑基因组生成网络连接,而非手动指定每条边。ES-HyperNEAT通过分析CPPN输出模式自动决定隐藏节点位置:递归使用四叉树细分空间,仅在CPPN输出方差高的区域扩展。这种自适应方法无需人工设定基底,拓展了基于NEAT的固定网格HyperNEAT框架。但四叉树难以张量化:各层级依赖父节点方差,强制串行评估;不同CPPN产生不同细分模式,无法批量处理;且叶节点数量不固定,违反JAX静态形状要求进行JIT编译。我们先前验证过,当深度超过5时性能受限,即使重实现也仅获得轻微提速。本文提出EMR-HyperNEAT,预先计算所有分辨率的所有位置,再统一用方差阈值筛选:将原ES-HyperNEAT的subdivide_if(var > θ)改为eval_all(); filter(var > θ)。虽多执行部分查询,但所有查询可并行于核心与种群成员之间,将复杂度从\BigO(4^D)降至\BigO(4^D/P),P为并行核心数。通过连接类型分类,实现循环基底配置。实验验证,在深度5-7的XOR任务上,设备端GPU提速12-34倍,各类基准测试求解率显著提升。
原文摘要 · Abstract (English)
In neuroevolution, indirect encoding generates neural network connectivity from a compact genome rather than specifying each connection. ES-HyperNEAT automatically discovers where to place hidden nodes by examining CPPN output patterns: it recursively subdivides space using a quadtree, expanding regions where CPPN outputs show high variance. This adaptive approach discovers network topology without manual substrate specification, extending the fixed-grid HyperNEAT framework built on NEAT. However, the quadtree resists tensorization. Each depth level depends on the parent's variance, forcing sequential evaluation. Different CPPNs produce different subdivision patterns, preventing batching. And variable leaf counts are incompatible with JAX's static shape requirement for JIT compilation. Our prior work confirmed these limits at depths exceeding 5, and a JAX reimplementation of the quadtree yielded only marginal speedup despite batched optimizations, motivating the eager reformulation presented here. We present EMR-HyperNEAT, which evaluates all positions at all resolutions up front, then filters using the same variance criterion: ES-HyperNEAT's subdivide_if(var > $θ$) becomes eval_all(); filter(var > $θ$). This performs more CPPN queries than necessary, but all queries become independent and parallelizable across both cores and population members, reducing complexity from \BigO($4^D$) to \BigO($4^D/P$) across $P$ parallel cores. Recurrent substrate configurations become feasible through a connection type taxonomy. The experiments section validates 12-34$\times$ on-device GPU speedup on XOR at depths 5-7, and empirically higher solve rates across benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。