SPADE通过分拆延迟特征，让Transformer更好生成高粒度粒子流数据。
SPADE: Split-and-Delay Embeddings for Autoregressive High-Granularity Calorimeter Simulation
- 将多特征令牌分开展示并错开输入时间，让自注意力捕捉内部相关性。
- 在ILD探测器光子簇射生成上，性能媲美当前最优模型AllShowers。
- 适用于任何含多特征的生成任务，可支持大模型预训练流程。
我们提出SPADE（SPlit And Delay Embeddings），一种针对携带多个特征的序列的自回归Transformer。不同于联合嵌入所有特征，SPADE对各特征独立嵌入，并通过延迟不同特征流的方式，使标准自注意力机制能够学习到单个令牌内的相关性。该方法应用于高粒度ILD探测器的点云簇射生成任务，在光子簇射生成上表现与当前最优模型AllShowers相当，显著优于其基于VQ-VAE的前身OmniJet-α_C。该机制可推广至任意具有多特征令牌的生成任务，支持更高维数据的大模型式预训练流程。
原文摘要 · Abstract (English)
We introduce SPADE (SPlit And Delay Embeddings), an autoregressive transformer for sequences whose tokens carry multiple features. Rather than embedding these features jointly, SPADE embeds them independently. Delaying each feature stream relative to the previous one allows intra-token correlations to be learned by the standard self-attention mechanism. Applied to point-cloud calorimeter shower generation in the highly granular ILD detector, SPADE is competitive with the state of the art AllShowers model on photon showers, and substantially outperforms its VQ-VAE-based predecessor OmniJet-$α_C$. The mechanism is applicable to any generative task with multi-feature tokens, enabling LLM-style pretraining workflows for higher-dimensional data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。