神经进化在迁移学习中表现优于强化学习,尤其在任务变化时更稳定。
When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks?
- 用模块化逻辑电路和可变机器人形态设计双基准测试
- 神经进化在多数任务上超越强化学习的迁移能力
- 适合研究鲁棒性智能体与可扩展算法的学者
持续高效地跨任务迁移技能是生物智能的特征,也是人工系统长期追求的目标。强化学习(RL)虽在高维控制任务中占主导地位,但对任务变化敏感且易出现灾难性遗忘。神经进化(NE)因其鲁棒性、可扩展性及跳出局部最优的能力近年受到关注。本文首次系统考察了NE在迁移学习中的潜力,提出了两个新基准:其一为‘步进门’,要求神经网络模拟逻辑电路,强调模块重复与变异;其二为‘ecorobot’,基于Brax物理引擎扩展墙体、障碍物等环境元素,并支持快速切换机器人形态。两个基准均包含渐进式难度的课程设置,用于评估技能迁移。实验表明,不同神经进化方法在迁移能力上差异显著,且常优于强化学习基线。结果支持将神经进化作为构建更适应性智能体的基础,并揭示其向复杂现实问题扩展的挑战。
原文摘要 · Abstract (English)
The ability to continuously and efficiently transfer skills across tasks is a hallmark of biological intelligence and a long-standing goal in artificial systems. Reinforcement learning (RL), a dominant paradigm for learning in high-dimensional control tasks, is known to suffer from brittleness to task variations and catastrophic forgetting. Neuroevolution (NE) has recently gained attention for its robustness, scalability, and capacity to escape local optima. In this paper, we investigate an understudied dimension of NE: its transfer learning capabilities. To this end, we introduce two benchmarks: a) in stepping gates, neural networks are tasked with emulating logic circuits, with designs that emphasize modular repetition and variation b) ecorobot extends the Brax physics engine with objects such as walls and obstacles and the ability to easily switch between different robotic morphologies. Crucial in both benchmarks is the presence of a curriculum that enables evaluating skill transfer across tasks of increasing complexity. Our empirical analysis shows that NE methods vary in their transfer abilities and frequently outperform RL baselines. Our findings support the potential of NE as a foundation for building more adaptable agents and highlight future challenges for scaling NE to complex, real-world problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。