用大模型自动完成科研全流程,加速算法优化与实验验证。
AlphaLab: Autonomous Multi-Agent Research Across Optimization Domains with Frontier LLMs

- 大模型自主分析数据、构建评估框架并运行大规模实验。
- 在GPU优化中提速4.4倍(最高91倍),预训练损失降低22%。
- 支持多模型协作,覆盖更广搜索空间,适合自动化科研场景。
我们提出AlphaLab,一个利用前沿大模型智能体能力的自治研究系统,可自动完成量化、计算密集型领域的完整实验流程。仅需提供数据集和自然语言目标,系统无需人工干预即可完成三个阶段:(1) 适应领域并探索数据,编写分析代码并生成研究报告;(2) 构建并对抗性验证自身评估框架;(3) 通过策略师/工作者循环在大规模GPU上运行实验,将领域知识积累到持久的作战手册中,实现在线提示优化。所有领域特异性行为由模型自动生成的适配器处理,同一管道可无修改应对质性不同的任务。我们在三个领域评估了使用GPT-5.2和Claude Opus 4.6的AlphaLab:CUDA内核优化中,生成的GPU内核平均比torch.compile快4.4倍(最高91倍);LLM预训练中,系统整体验证损失比单次基线低22%;交通预测中,经研究并实现文献中的模型家族后,性能优于标准基线23-25%。两种模型在每个领域均发现不同解决方案(无一方始终占优),表明多模型协同可提供互补搜索覆盖。附录还报告了金融时间序列预测结果,并已开源全部代码至https://brendanhogan.github.io/alphalab-paper/。
原文摘要 · Abstract (English)
We present AlphaLab, an autonomous research harness that leverages frontier LLM agentic capabilities to automate the full experimental cycle in quantitative, computation-intensive domains. Given only a dataset and a natural-language objective, AlphaLab proceeds through three phases without human intervention: (1) it adapts to the domain and explores the data, writing analysis code and producing a research report; (2) it constructs and adversarially validates its own evaluation framework; and (3) it runs large-scale GPU experiments via a Strategist/Worker loop, accumulating domain knowledge in a persistent playbook that functions as a form of online prompt optimization. All domain-specific behavior is factored into adapters generated by the model itself, so the same pipeline handles qualitatively different tasks without modification. We evaluate AlphaLab with two frontier LLMs (GPT-5.2 and Claude Opus 4.6) on three domains: CUDA kernel optimization, where it writes GPU kernels that run 4.4x faster than torch.compile on average (up to 91x); LLM pretraining, where the full system achieves 22% lower validation loss than a single-shot baseline using the same model; and traffic forecasting, where it beats standard baselines by 23-25% after researching and implementing published model families from the literature. The two models discover qualitatively different solutions in every domain (neither dominates uniformly), suggesting that multi-model campaigns provide complementary search coverage. We additionally report results on financial time series forecasting in the appendix, and release all code at https://brendanhogan.github.io/alphalab-paper/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。