arXiv:2504.13541cs.NEcs.AI2025-04中稿 · the 63rd ACM/IEEE …被引 1

用自适应切换策略让神经脉冲网络同时学多个任务,减少干扰、省电且性能更强。

Scalable Multi-Task Learning through Spiking Neural Networks with Adaptive Task-Switching Policy for Intelligent Autonomous Agents

  • 基于带激活树突和双结构的脉冲网络,用任务信号生成专用子网络。
  • 在Atari游戏上表现优于现有方法,如'Enduro'得分355.2,'Pong'得-8.8。
  • 无需增加复杂度,适合资源受限的智能自主系统长期多任务运行。

资源受限的自主代理同时学习多个任务对适应多样化现实环境至关重要。现有强化学习方法仍受任务干扰影响,性能不佳。虽然先进工作使用脉冲神经网络(SNN)提升多任务学习并实现低功耗运行,但依赖固定任务切换周期,限制了性能与可扩展性。为此,我们提出SwitchMT:一种采用自适应任务切换策略的新型多任务学习方法。核心包括:(1) 使用带激活树突和双结构的深度脉冲Q网络,通过任务特定上下文信号构建专用子网络;(2) 设计结合奖励与网络内部动态的自适应任务切换策略。实验表明,SwitchMT在多个Atari游戏中表现优异,如'Pong'得分为-8.8,'Breakout'为5.6,'Enduro'达355.2,且游戏持续时间更长。结果验证了该方法有效缓解任务干扰,不增加网络复杂度,使智能自主代理具备可扩展的多任务学习能力。

原文摘要 · Abstract (English)

Training resource-constrained autonomous agents on multiple tasks simultaneously is crucial for adapting to diverse real-world environments. Recent works employ reinforcement learning (RL) approach, but they still suffer from sub-optimal multi-task performance due to task interference. State-of-the-art works employ Spiking Neural Networks (SNNs) to improve RL-based multi-task learning and enable low-power/energy operations through network enhancements and spike-driven data stream processing. However, they rely on fixed task-switching intervals during its training, thus limiting its performance and scalability. To address this, we propose SwitchMT, a novel methodology that employs adaptive task-switching for effective, scalable, and simultaneous multi-task learning. SwitchMT employs the following key ideas: (1) leveraging a Deep Spiking Q-Network with active dendrites and dueling structure, that utilizes task-specific context signals to create specialized sub-networks; and (2) devising an adaptive task-switching policy that leverages both rewards and internal dynamics of the network parameters. Experimental results demonstrate that SwitchMT achieves competitive scores in multiple Atari games (i.e., Pong: -8.8, Breakout: 5.6, and Enduro: 355.2) and longer game episodes as compared to the state-of-the-art. These results also highlight the effectiveness of SwitchMT methodology in addressing task interference without increasing the network complexity, enabling intelligent autonomous agents with scalable multi-task learning capabilities.

脉冲神经网络多任务学习自适应切换强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。