用大模型生成新模型组件，自动加速机器学习创新
Towards Automated Machine Learning Research
- 利用大模型跨领域知识提出新型组件，突破预设组件限制
- 结合奖励模型筛选高潜力假设，提升创新效率
- 适合关注自动化机器学习与模型设计的科研人员
本文探索一种自上而下的自动化机器学习研究方法，通过组件级创新实现增量进步，借助大语言模型（LLMs）推动。该框架系统性地生成新组件，验证其可行性，并评估其性能。与传统AutoML和NAS依赖预定义硬编码组件的自下而上组合搜索不同，本方法利用大模型中嵌入的跨领域知识，提出可能超出预设集合的新组件。通过引入奖励模型优先筛选有前景的假设，旨在提升假设生成与评估过程的效率。我们希望这一方法为探索提供新路径，并推动该领域的持续对话。
原文摘要 · Abstract (English)
This paper explores a top-down approach to automating incremental advances in machine learning research through component-level innovation, facilitated by Large Language Models (LLMs). Our framework systematically generates novel components, validates their feasibility, and evaluates their performance against existing baselines. A key distinction of this approach lies in how these novel components are generated. Unlike traditional AutoML and NAS methods, which often rely on a bottom-up combinatorial search over predefined, hardcoded base components, our method leverages the cross-domain knowledge embedded in LLMs to propose new components that may not be confined to any hard-coded predefined set. By incorporating a reward model to prioritize promising hypotheses, we aim to improve the efficiency of the hypothesis generation and evaluation process. We hope this approach offers a new avenue for exploration and contributes to the ongoing dialogue in the field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。