给分子语言模型加局部记忆，提升生成与预测效果
Augmenting Molecular Language Models with Local $n$-gram Memory
- 用n-gram记忆模块捕捉分子字符串的局部化学模式
- 在三种任务中均优于基线，参数多3倍仍更高效
- 适合分子生成、反应预测等药物研发场景
基于Transformer的SMILES字符串语言模型存在局部性差距：标准字符级分词会破坏化学上有意义的片段，迫使模型反复学习局部语法，牺牲长程依赖建模。为解决此问题而不干扰标准分词器，我们提出MolGram，将条件n-gram记忆模块融入分子语言模型。MolGram通过可扩展哈希查找将局部字符串模式映射为学习到的嵌入，并动态将其区域上下文注入隐藏状态。在三个任务（包括无条件分子生成、正向反应预测和单步逆合成）上的评估显示，MolGram持续提升性能。关键分析表明，即使参数量是基线的3倍，MolGram仍表现更优，证明显式局部模式记忆是一种高效归纳偏置。
原文摘要 · Abstract (English)
Transformer-based language models for SMILES strings suffer from a locality gap: standard character-level tokenization fragments chemically meaningful motifs, forcing models to repeatedly learn local syntax at the expense of long-range dependencies. To address this without disrupting standard tokenizers, we propose MolGram, which integrates a conditional $n$-gram memory module into molecular language models. MolGram maps local string patterns to learned embeddings via scalable hash lookups and dynamically injects this regional context into hidden states. Evaluations across three tasks, including unconditional molecule generation, forward reaction prediction, and single-step retrosynthesis, show that MolGram consistently improves performance. Crucially, our analyses demonstrate that MolGram outperforms baselines with 3$\times$ more parameters, establishing explicit local pattern memory as a highly efficient inductive bias.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。