用分组生成提升Transformer采样效率，可处理180×180自旋系统
Sampling two-dimensional spin systems with transformers
- 每次生成一组自旋而非单个，提升采样并行度
- 在128×128伊辛模型上有效样本量达基准20倍
- 适用于大尺寸自旋系统建模，尤其适合物理模拟
基于密集层或卷积层的自回归神经网络已被证明是生成经典自旋系统的一种可行方法。与之不同的是，通常认为变压器（transformer）在采样中计算效率较低。本文提出一种新型基于变压器的神经采样方法：每次生成一组自旋而非单个自旋，并构建近似概率模型以进一步提高算法效率。尽管该方法计算开销高于密集网络或基于CNN的方法，但在伊辛模型下仍成功实现了最大180×180自旋系统的采样。当训练于128×128伊辛模型且处于临界温度时，采样器的有效样本量约为之前最先进神经采样器的20倍。此外，我们在二维爱德华-安德森模型上测试了该算法，成功训练了64×64自旋系统。
原文摘要 · Abstract (English)
Autoregressive Neural Networks based on dense or convolutional layers have recently been shown to be a viable strategy for generating classical spin systems. Unlike these methods, sampling with transformers is commonly considered to be computationally inefficient. In this work, we propose a novel approach to transformer-based neural samplers in which we generate not a single spin per step but groups of spins. As an additional improvement, we construct a model of approximated probabilities, further improving the efficiency of the algorithm. Despite our approach being computationally heavier than dense networks or CNN-based approaches, we were able to sample larger systems of up to $180 \times 180$ spins in case of the Ising model. The Effective Sample Size of our sampler is $\sim 20$ times larger than that of the previous state-of-the-art neural sampler when trained for the $128 \times 128$ Ising model at critical temperature. Finally, we also test our algorithm on the 2D Edwards-Anderson model, where we train $64\times 64$ spin systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。