用自监督学习构建高能对撞机物理基础模型，提升喷注分类性能。
HEP-JEPA: A foundation model for collider physics using joint embedding predictive architecture
- 基于联合嵌入预测架构，利用喷注部分子信息自监督预训练
- 在1亿喷注数据上训练，下游任务准确率超越现有模型
- 适合高能物理研究者、粒子探测算法开发者使用
我们提出一种基于Transformer架构的高能粒子对撞机基础模型，用于大型强子对撞机等场景的任务。模型采用受联合嵌入预测架构启发的自监督策略进行训练，利用包含1亿个已知粒子喷注的JetClass数据集，以数据为中心的方法让模型通过部分子作为上下文，预测未见目标部分子的嵌入表示。预训练模型在标准分类基准任务中表现优异。我们在两个下游任务上测试：顶夸克标记与轻夸克喷注和胶子喷注区分。通过任务特定指标与基线对比，模型性能优于高能物理领域当前最先进方法。
原文摘要 · Abstract (English)
We present a transformer architecture-based foundation model for tasks at high-energy particle colliders such as the Large Hadron Collider. We train the model to classify jets using a self-supervised strategy inspired by the Joint Embedding Predictive Architecture. We use the JetClass dataset containing 100M jets of various known particles to pre-train the model with a data-centric approach -- the model uses a fraction of the jet constituents as the context to predict the embeddings of the unseen target constituents. Our pre-trained model fares well with other datasets for standard classification benchmark tasks. We test our model on two additional downstream tasks: top tagging and differentiating light-quark jets from gluon jets. We also evaluate our model with task-specific metrics and baselines and compare it with state-of-the-art models in high-energy physics. Project site: https://hep-jepa.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。