用强化学习引导大模型生成更相关、可解释的分子属性,提升预测性能。
AttriLens-Mol: Attribute Guided Reinforcement Learning for Molecular Property Prediction with Large Language Models
- 通过属性引导的奖励机制,让大模型推理更聚焦
- 在4000样本上训练,性能媲美甚至超越顶尖模型
- 提取的属性可直接用于解释性建模,效果更优
大型语言模型(LLMs)在分子性质预测中展现出潜力,但常依赖人工设计的提示和思维链模板。尽管如DeepSeek-R1等先进模型采用强化学习扩展推理过程,其输出仍可能冗长且不相关。本文提出AttriLens-Mol,一种基于属性引导的强化学习框架。该框架通过三类奖励机制引导模型:(1) 格式奖励,鼓励生成结构化的属性输出;(2) 计数奖励,抑制无关属性的枚举;(3) 合理性奖励,利用高级大模型与RDKit验证生成属性的相关性。该方法在推理过程中隐式激发模型对相关分子属性的知识,从而更有效地进行性质预测。在分布内与分布外数据集上的实验表明,仅用4,000个样本训练7B规模的R1-Distilled-Qwen2.5和R1-Distilled-LLaMA3.1模型,性能显著提升,达到或超过监督微调模型(如Mol-Instructions、ChemDFM)及先进模型(GPT-3.5、GPT-4o、DeepSeek-V3、DeepSeek-R1)。此外,所提取的属性作为特征输入可解释决策树模型时,表现优于传统提示法生成的属性,证明AttriLens-Mol能有效挖掘更相关、更具预测性的分子属性,增强预测性能与可解释性。代码已开源:https://github.com/szu-tera/AttriLens-Mol。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown promise in assisting molecular property prediction tasks but often rely on human-crafted prompts and chain-of-thought templates. While recent advanced large reasoning models like DeepSeek-R1 employ reinforcement learning for an extended ``thinking'' process, their reasoning can be verbose and lack relevance. We introduce AttriLens-Mol, an attribute-guided reinforcement learning framework for molecular property prediction with LLMs. AttriLens-Mol steers the model's reasoning by using: (1) a format reward encouraging attribute-based structured output, (2) a count reward to avoid enumerating irrelevant attributes, and (3) a rationality reward using advanced LLMs and RDKit to verify the relatedness of the generated attributes. This approach implicitly elicits the model's inherent knowledge of relevant molecular attributes during reasoning, enables making predictions for the molecular property more effectively. Experiments on both in-distribution and out-of-distribution datasets show that, training both 7B-size R1-Distilled-Qwen2.5 and R1-Distilled-LLaMA3.1 models on 4,000 samples with our proposed AttriLens-Mol method significantly boosts the performance, getting comparable or better results than supervised fine-tuning models (Mol-Instructions, ChemDFM, etc.) and advanced models (GPT-3.5, GPT-4o, DeepSeek-V3, DeepSeek-R1, etc.). Further, our extracted attributes for the target property, when used as features for an interpretable decision tree model, yield superior performance compared to attributes generated by prompting LLMs. This shows that AttriLens-Mol effectively elicits more relevant and predictive molecular attributes, leading to enhanced interpretability and performance for property prediction. We release the code in https://github.com/szu-tera/AttriLens-Mol.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。