对比多种元学习方法，为强化学习算法设计提供实证指导
How Should We Meta-Learn Reinforcement Learning Algorithms?
- 用演化算法和大模型生成代码，自动优化强化学习算法
- 评估了不同方法在性能、样本成本和训练时间上的表现差异
- 提出可操作指南，帮助设计更高效的新强化学习算法
从数据中元学习算法替代人工设计，正成为提升机器学习系统性能的流行范式。元学习在强化学习（RL）中尤为有前景，因现有算法常源自监督或无监督学习，不适用于RL任务。然而，此前缺乏对不同元学习方法的系统性比较，例如使用演化算法优化黑箱函数，或利用大模型生成代码。本文针对强化学习流程的不同环节，对多种元学习算法进行实证比较。除元训练与元测试性能外，还考察了可解释性、样本成本和训练时间等关键因素。基于这些发现，本文提出若干元学习新强化学习算法的实践指南，以确保未来学习到的算法具备最优性能。
原文摘要 · Abstract (English)
The process of meta-learning algorithms from data, instead of relying on manual design, is growing in popularity as a paradigm for improving the performance of machine learning systems. Meta-learning shows particular promise for reinforcement learning (RL), where algorithms are often adapted from supervised or unsupervised learning despite their suboptimality for RL. However, until now there has been a severe lack of comparison between different meta-learning algorithms, such as using evolution to optimise over black-box functions or LLMs to propose code. In this paper, we carry out this empirical comparison of the different approaches when applied to a range of meta-learned algorithms which target different parts of the RL pipeline. In addition to meta-train and meta-test performance, we also investigate factors including the interpretability, sample cost and train time for each meta-learning algorithm. Based on these findings, we propose several guidelines for meta-learning new RL algorithms which will help ensure that future learned algorithms are as performant as possible.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。