53K参数小模型学会分子语法，生成有效分子结构超95%。
SMolLM: Small Language Models Learn Small Molecular Grammar
- 用53K参数共享权重的Transformer逐层解析分子式约束。
- 在ZINC-250K上生成95%有效SMILES，优于10倍参数的GPT。
- 可解释性强，适合研究化学语言的迭代计算机制。
用于分子设计的语言模型已扩展至数亿参数，但其学习化学语法规则的过程仍不清晰。我们训练了仅53K参数的SMolLM（一个权重共享的Transformer），在ZINC-250K类药物分子基准上生成的SMILES结构有95%有效，性能超越参数量达十倍的标准GPT模型。机制分析显示，同一网络模块按固定层级顺序处理约束：先匹配括号，再处理环，最后校验价态。通过错误分类与线性探针验证，并经消融实验确认括号匹配头的关键作用。这些结果构建了一个紧凑且机制可解释的分子生成器，也为形式语言领域的迭代计算研究提供了测试平台。
原文摘要 · Abstract (English)
Language models for molecular design have scaled to hundreds of millions of parameters, yet how they learn chemical grammar is poorly understood. We train SMolLM, a 53K-parameter weight-shared transformer, to generate novel SMILES with 95% validity on the ZINC-250K drug-like-molecule benchmark, outperforming a standard GPT with 10 times more parameters. Mechanistically, the same block resolves SMILES constraints across passes in a fixed hierarchy: brackets first, rings second, and valence last, as shown by error classification and linear probing, with ablation isolating the bracket-matching head. Together, these results yield a compact, mechanistically interpretable molecular generator and a testbed for studying iterative computation in formal-language domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。