构建可扩展的核与粒子物理基础模型，实现跨任务高效泛化。
FM4NPP: A Scaling Foundation Model for Nuclear and Particle Physics
- 基于1100万事件数据，提出新型自监督训练方法。
- 模型达1.88亿参数，适配任务表现优于基线。
- 仅用线性映射即可适配不同下游任务，数据效率高。
大型语言模型通过自监督学习实现了大规模通用模型的革命性发展，这一范式推动了科学基础模型（FMs）的兴起。然而，实验粒子物理中的探测器数据稀疏且空间分布离散，与自然语言差异显著，难以直接应用。本文探讨粒子物理基础模型的可扩展性与泛化能力。我们构建了一个包含超过1100万粒子碰撞事件的新数据集，并设计了一套下游任务及标注数据用于评估。提出一种针对探测器数据的新型自监督训练方法，验证了神经网络在高达1.88亿参数规模下的可扩展性。采用冻结权重+任务特定适配器的方式，该基础模型在所有下游任务中均显著优于基线模型，且表现出优异的数据效率。进一步分析表明，模型提取的表征具有任务无关性，仅通过单一线性映射即可适应不同任务。
原文摘要 · Abstract (English)
Large language models have revolutionized artificial intelligence by enabling large, generalizable models trained through self-supervision. This paradigm has inspired the development of scientific foundation models (FMs). However, applying this capability to experimental particle physics is challenging due to the sparse, spatially distributed nature of detector data, which differs dramatically from natural language. This work addresses if an FM for particle physics can scale and generalize across diverse tasks. We introduce a new dataset with more than 11 million particle collision events and a suite of downstream tasks and labeled data for evaluation. We propose a novel self-supervised training method for detector data and demonstrate its neural scalability with models that feature up to 188 million parameters. With frozen weights and task-specific adapters, this FM consistently outperforms baseline models across all downstream tasks. The performance also exhibits robust data-efficient adaptation. Further analysis reveals that the representations extracted by the FM are task-agnostic but can be specialized via a single linear mapping for different downstream tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。