用程序生成海量机器学习算法任务，推动自动化算法发现
DiscoGen: Procedural Generation of Algorithm Discovery Tasks in Machine Learning
- 通过少量参数生成数十亿种不同难度的机器学习任务
- 构建固定评估集DiscoBench，支持算法发现代理的可靠测试
- 适合研究自动化算法设计与强化学习优化的研究者
自动化机器学习算法开发有望带来新突破，但现有任务集存在评估方法不当、数据污染及问题重复率高等问题。为此，我们提出DiscoGen，一种用于生成机器学习算法发现任务的程序化工具，可生成优化强化学习策略或图像分类损失函数等任务。受强化学习中程序化生成的成功启发，DiscoGen 能覆盖数十亿种不同复杂度和难度的任务，仅需少量配置参数即可定义。这些任务可用于训练和优化算法发现代理（ADAs）。我们进一步提出DiscoBench，一个固定的小型基准任务集，用于对ADAs进行严谨评估。最后，我们展示了DiscoGen在自动提示调优中的规模化应用，并指出了若干前沿研究方向。代码已开源：https://github.com/AlexGoldie/discogen。
原文摘要 · Abstract (English)
Automating the development of machine learning algorithms has the potential to unlock new breakthroughs. However, our ability to improve and evaluate algorithm discovery systems has thus far been limited by existing task suites. They suffer from many issues, such as: poor evaluation methodologies; data contamination; and containing saturated or very similar problems. Here, we introduce DiscoGen, a procedural generator of algorithm discovery tasks for machine learning, such as developing optimisers for reinforcement learning or loss functions for image classification. Motivated by the success of procedural generation in reinforcement learning, DiscoGen spans billions of tasks of varying difficulty and complexity from a range of machine learning fields. These tasks are specified by a small number of configuration parameters and can be used to optimise algorithm discovery agents (ADAs). We present DiscoBench, a fixed, small subset of DiscoGen tasks for principled evaluation of ADAs. Finally, we propose a number of ambitious, impactful research directions enabled by DiscoGen, and demonstrate its use for ADA optimisation through scaling experiments for automated prompt tuning. DiscoGen is released open-source at https://github.com/AlexGoldie/discogen.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。