用智能体强化学习优化分子结构,实现化学有效且可解释的迭代改进。
MolAct: An Agentic RL Framework for Molecular Editing and Property Optimization
- 构建双阶段训练框架,让大模型学会结合推理与化学工具进行分子编辑
- 7B模型在添加、删除、替换操作中达到95%-100%有效率,超越多个闭源模型
- 适用于药物研发等需精准调控分子性质的场景,兼具可靠性和可解释性
分子编辑与优化是多步问题,需在保持化学有效性与结构相似性的前提下逐步提升性质。我们将其建模为序列化、工具引导的决策过程,提出MolAct——首个将分子设计形式化为智能体强化学习问题的框架。该框架采用两阶段训练:先建立编辑能力,再复用已有行为进行性质优化。模型通过多轮交互调用化学工具(如有效性验证、性质评估、相似性控制),并利用反馈优化后续修改。我们实现了两个模型家族:MolEditAgent用于分子编辑,MolOptAgent用于分子优化。在编辑任务中,MolEditAgent-7B在添加、删除、替换操作中分别实现100%、95%、98%的有效率,优于DeepSeek-R1等强闭源“思考”基线;3B版本接近Qwen3-32B-think等更大规模开源模型表现。在优化任务中,基于MolEditAgent-7B训练的MolOptAgent-7B在LogP指标上超越最佳闭源基线(如Claude 3.7),在溶解度等其他目标上保持均衡表现。结果表明,将分子设计视为多步、工具增强的过程,是实现可靠且可解释改进的关键。
原文摘要 · Abstract (English)
Molecular editing and optimization are multi-step problems that require iteratively improving properties while keeping molecules chemically valid and structurally similar. We frame both tasks as sequential, tool-guided decisions and introduce MolAct, an agentic reinforcement learning framework that employs a two-stage training paradigm: first building editing capability, then optimizing properties while reusing the learned editing behaviors. To the best of our knowledge, this is the first work to formalize molecular design as an Agentic Reinforcement Learning problem, where an LLM agent learns to interleave reasoning, tool-use, and molecular optimization. The framework enables agents to interact in multiple turns, invoking chemical tools for validity checking, property assessment, and similarity control, and leverages their feedback to refine subsequent edits. We instantiate the MolAct framework to train two model families: MolEditAgent for molecular editing tasks and MolOptAgent for molecular optimization tasks. In molecular editing, MolEditAgent-7B delivers 100, 95, and 98 valid add, delete, and substitute edits, outperforming strong closed "thinking" baselines such as DeepSeek-R1; MolEditAgent-3B approaches the performance of much larger open "thinking" models like Qwen3-32B-think. In molecular optimization, MolOptAgent-7B (trained on MolEditAgent-7B) surpasses the best closed "thinking" baseline (e.g., Claude 3.7) on LogP and remains competitive on solubility, while maintaining balanced performance across other objectives. These results highlight that treating molecular design as a multi-step, tool-augmented process is key to reliable and interpretable improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。