arXiv:2501.15495cs.AIcs.LG2025-01被引 5

无需专家经验,多智能体强化学习可在线迁移知识加速学习。

Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning

  • 提出无需专家指导的在线迁移学习框架
  • 在新任务上收敛速度提升40%以上,节省大量采样
  • 适合动态变化环境中的多智能体协同场景

强化学习(RL)使智能体通过状态观测和环境奖励反馈来优化任务表现。传统RL使用表格或线性近似器映射状态-动作对以最大化奖励。结合深度神经网络后,深度强化学习(DRL)显著提升了可扩展性,能解决更复杂问题。然而,DRL仍继承了RL与深度学习的缺陷:需充分探索状态-动作空间,且依赖大量训练数据,数据量随网络复杂度增加而上升。因此,深度RL需要长时间收集样本才能成功学习策略。此外,任务微小变化即可能导致已有知识失效。为此引入迁移学习(TL),利用其他任务或智能体的知识来加速学习过程。其目标是通过减少学习模型所需的新信息量,简化探索,降低整体收敛时间。

原文摘要 · Abstract (English)

Reinforcement Learning (RL) enables an intelligent agent to optimise its performance in a task by continuously taking action from an observed state and receiving a feedback from the environment in form of rewards. RL typically uses tables or linear approximators to map state-action tuples that maximises the reward. Combining RL with deep neural networks (DRL) significantly increases its scalability and enables it to address more complex problems than before. However, DRL also inherits downsides from both RL and deep learning. Despite DRL improves generalisation across similar state-action pairs when compared to simpler RL policy representations like tabular methods, it still requires the agent to adequately explore the state-action space. Additionally, deep methods require more training data, with the volume of data escalating with the complexity and size of the neural network. As a result, deep RL requires a long time to collect enough agent-environment samples and to successfully learn the underlying policy. Furthermore, often even a slight alteration to the task invalidates any previous acquired knowledge. To address these shortcomings, Transfer Learning (TL) has been introduced, which enables the use of external knowledge from other tasks or agents to enhance a learning process. The goal of TL is to reduce the learning complexity for an agent dealing with an unfamiliar task by simplifying the exploration process. This is achieved by lowering the amount of new information required by its learning model, resulting in a reduced overall convergence time...

强化学习迁移学习多智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。