arXiv:2601.14732cs.CVcs.CL2026-01

用视觉与几何信息融合建模分子,提升生成准确性与物理合理性。

DeepMoLM: Leveraging Visual and Geometric Structural Information for Molecule-Text Modeling

  • 双视角融合图像与分子构象几何特征,不依赖原子坐标。
  • 在PubChem图文描述任务上相对基线提升12.3%的METEOR得分。
  • 适合需要精确化学结构生成的药物发现与文献挖掘场景。

用于药物发现和化学文献挖掘的AI模型需理解分子图像并生成符合三维几何与立体化学的输出。现有分子语言模型多依赖字符串或图结构,而视觉语言模型常忽略立体化学细节,难以将连续3D结构映射为离散标记。我们提出DeepMoLM:一种双视角框架,将高分辨率分子图像与基于分子构象的几何不变量对齐。DeepMoLM保留1024×1024输入中的高频信息,将构象邻域编码为离散的扩展三维指纹(Extended 3-Dimensional Fingerprints),并通过交叉注意力融合视觉与几何流,实现无需原子坐标的物理合理生成。在PubChem图文描述任务中,相比最强通用基线,其METEOR得分相对提升12.3%,同时在专用设置下对属性查询生成有效数值,分子量预测平均绝对误差(MAE)为13.64 g/mol,复杂度预测为37.89。在ChEBI-20图像描述生成任务中,超越通用基线并达到当前最优视觉语言模型水平。代码已开源:https://github.com/1anj/DeepMoLM。

原文摘要 · Abstract (English)

AI models for drug discovery and chemical literature mining must interpret molecular images and generate outputs consistent with 3D geometry and stereochemistry. Most molecular language models rely on strings or graphs, while vision-language models often miss stereochemical details and struggle to map continuous 3D structures into discrete tokens. We propose DeepMoLM: Deep Molecular Language M odeling, a dual-view framework that grounds high-resolution molecular images in geometric invariants derived from molecular conformations. DeepMoLM preserves high-frequency evidence from 1024 $\times$ 1024 inputs, encodes conformer neighborhoods as discrete Extended 3-Dimensional Fingerprints, and fuses visual and geometric streams with cross-attention, enabling physically grounded generation without atom coordinates. DeepMoLM improves PubChem captioning with a 12.3% relative METEOR gain over the strongest generalist baseline while staying competitive with specialist methods. It produces valid numeric outputs for all property queries and attains MAE 13.64 g/mol on Molecular Weight and 37.89 on Complexity in the specialist setting. On ChEBI-20 description generation from images, it exceeds generalist baselines and matches state-of-the-art vision-language models. Code is available at https://github.com/1anj/DeepMoLM.

分子建模视觉语言几何结构药物发现

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。