用大模型团队自动完成粒子物理数据分析,效果接近人类顶尖水平。
Agents of Discovery
- 构建多个大模型代理协同工作,像人一样写代码调用工具完成分析。
- 在LHC奥运会数据集上,最佳方案性能达到人类最先进水平。
- 适合想探索自动化科研流程的研究者或工具开发者。
现代粒子物理等基础研究面临海量数据,亟需更复杂的分析工具与流程。尽管机器学习工具已广泛应用,但多为针对特定任务的专用算法,依赖编码的物理知识以达最优。本文探索一条新路径:利用大语言模型(LLMs)构建一组分工协作的智能体,模拟人类研究人员的行为——编写代码调用标准工具与库(包括机器学习系统),并基于前序迭代结果持续优化。若成功,此类代理系统可自动化处理常规分析环节,应对现代分析链日益增长的复杂性。我们以公开且广受研究的LHC奥运会数据集为任务,测试了多家主流商用LLM(OpenAI的GPT-4o、o4-mini、GPT-4.1和GPT-5)的表现及其稳定性。结果显示,该代理系统具备解决此数据分析问题的能力,最优方案性能与人类当前最先进结果相当。
原文摘要 · Abstract (English)
The substantial data volumes encountered in modern particle physics and other domains of fundamental physics research allow (and require) the use of increasingly complex data analysis tools and workflows. While the use of machine learning (ML) tools for data analysis has recently proliferated, these tools are typically special-purpose algorithms that rely, for example, on encoded physics knowledge to reach optimal performance. In this work, we investigate a new and orthogonal direction: Using recent progress in large language models (LLMs) to create a team of agents -- instances of LLMs with specific subtasks -- that jointly solve data analysis-based research problems in a way similar to how a human researcher might: by creating code to operate standard tools and libraries (including ML systems) and by building on results of previous iterations. If successful, such agent-based systems could be deployed to automate routine analysis components to counteract the increasing complexity of modern tool chains. To investigate the capabilities of current-generation commercial LLMs, we consider the task of anomaly detection via the publicly available and highly-studied LHC Olympics dataset. Several current models by OpenAI (GPT-4o, o4-mini, GPT-4.1, and GPT-5) are investigated and their stability tested. Overall, we observe the capacity of the agent-based system to solve this data analysis problem. The best agent-created solutions mirror the performance of human state-of-the-art results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。