轻量级扩散Transformer，高效生成晶体结构。
Crystalite: A Lightweight Transformer for Efficient Crystal Modeling
- 用原子级粒度编码替代传统独热编码，提升连续扩散效率。
- 引入几何增强模块，直接注入周期性最小图像对几何信息。
- 生成速度远超同类方法，且晶体结构预测性能领先。
晶体材料的生成模型通常依赖等变图神经网络，虽能良好捕捉几何结构，但训练成本高、采样慢。本文提出Crystalite，一种基于两种简单归纳偏置的轻量级扩散Transformer。其一为子原子分词（Subatomic Tokenization），以紧凑的化学结构化原子表示替代高维独热编码，更适配连续扩散过程；其二为几何增强模块（GEM），通过加性几何偏置将周期性最小图像对的几何信息直接注入注意力机制。二者结合使标准Transformer在保持简洁高效的同时，更契合晶体材料结构特性。Crystalite在晶体结构预测基准上达到当前最优表现，且在从头生成任务中取得最佳S.U.N.发现得分，采样速度显著快于依赖几何信息的主流方法。
原文摘要 · Abstract (English)
Generative models for crystalline materials often rely on equivariant graph neural networks, which capture geometric structure well but are costly to train and slow to sample. We present Crystalite, a lightweight diffusion Transformer for crystal modeling built around two simple inductive biases. The first is Subatomic Tokenization, a compact chemically structured atom representation that replaces high-dimensional one-hot encodings and is better suited to continuous diffusion. The second is the Geometry Enhancement Module (GEM), which injects periodic minimum-image pair geometry directly into attention through additive geometric biases. Together, these components preserve the simplicity and efficiency of a standard Transformer while making it better matched to the structure of crystalline materials. Crystalite achieves state-of-the-art results on crystal structure prediction benchmarks, and de novo generation performance, attaining the best S.U.N. discovery score among the evaluated baselines while sampling substantially faster than geometry-heavy alternatives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。