发现分子图特征层级影响模型表现，建议动态处理多级图信息
Exploring Hierarchical Molecular Graph Representation in Multimodal LLMs
- 分析不同粒度的分子图特征对模型的影响
- 单个图特征令牌不影响性能，但不同任务需不同层级特征
- 提示未来模型应动态处理多级图结构，适合药物研发研究者
随着大语言模型（LLM）和多模态模型的发展，越来越多研究将LLM应用于生化任务。通过结合图特征与分子文本表示，LLM可完成化学反应结果预测、分子性质描述等任务。然而，现有工作普遍忽视了图模态的多层次特性，而不同化学任务可能受益于不同层次的特征。本文首先研究特征粒度的影响，发现即使将所有GNN生成的特征令牌压缩为单一令牌，模型性能也未显著下降。随后，我们考察多种图特征层级，发现生成分子的质量及跨任务模型表现均依赖于不同的图特征层级。因此得出两个关键结论：(1) 当前分子相关多模态LLM对图特征的理解不完整；(2) 静态处理无法满足层级化图特征的需求。我们详细分享研究成果，旨在推动社区发展更先进的多模态LLM以融合分子图数据。
原文摘要 · Abstract (English)
Following the milestones in large language models (LLMs) and multimodal models, we have seen a surge in applying LLMs to biochemical tasks. Leveraging graph features and molecular text representations, LLMs can tackle various tasks, such as predicting chemical reaction outcomes and describing molecular properties. However, most current work overlooks the *multi-level nature* of the graph modality, even though different chemistry tasks may benefit from different feature levels. In this work, we first study the effect of feature granularity and reveal that even reducing all GNN-generated feature tokens to a single one does not significantly impact model performance. We then investigate the effect of various graph feature levels and demonstrate that both the quality of LLM-generated molecules and model performance across different tasks depend on different graph feature levels. Therefore, we conclude with two key insights: (1) current molecular-related multimodal LLMs lack a comprehensive understanding of graph features, and (2) static processing is not sufficient for hierarchical graph feature. We share our findings in detail, with the hope of paving the way for the community to develop more advanced multimodal LLMs for incorporating molecular graphs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。