arXiv:2507.21184cs.LGcs.AI2025-07被引 11

用大模型自动发现性能扩展规律,比人类手调更准。

Can Language Models Discover Scaling Laws?

  • 设计进化型代理SLDAgent,同步优化模型与参数
  • 在8个任务中均超越人工发现的扩展规律
  • 适合研究者和自动化科研工具开发者参考

发现预测模型规模性能的扩展规律是基础性难题,传统依赖耗时的人工实验。本文收集超过5000项实验,构建8个多样化的扩展规律发现任务。现有智能体难以准确生成规律公式,为此提出基于进化的SLDAgent,可自主探索变量间复杂关系。首次证明,SLDAgent能自动发现比人工推导更优的扩展规律,在所有任务中实现更精准的外推。通过深入分析,揭示其优越性来源,并验证其在预训练与微调中的实用价值。本工作确立了智能体科学发现新范式,表明AI可理解自身扩展行为,并向研究社区贡献新颖且可用的知识。

原文摘要 · Abstract (English)

Discovering scaling laws for predicting model performance at scale is a fundamental and open-ended challenge, mostly reliant on slow, case specific human experimentation. To investigate the potential for LLMs to automate this process, we collect over 5,000 experiments from existing literature and curate eight diverse scaling law discovery tasks. While existing agents struggle to produce accurate law formulas, this paper introduces SLDAgent, an evolution-based agent that co-optimize the scaling law model and the parameters, enabling it to autonomously explore complex relationships between variables. For the first time, we demonstrates that SLDAgent can automatically discover laws that exhibit consistently more accurate extrapolation than their established, human-derived counterparts across all tasks. Through comprehensive analysis, we elucidate why these discovered laws are superior and verify their practical utility in both pretraining and finetuning applications. This work establishes a new paradigm for agentic scientific discovery, showing that AI systems can understand their own scaling behavior, and can contribute novel and practical knowledge back to the research community.

大模型扩展规律智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。