用记忆增强强化学习,让药物分子优化更省试错次数
MolMem: Memory-Augmented Agentic Reinforcement Learning for Sample-Efficient Molecular Optimization

- 引入双记忆系统:静态示例库+动态技能库,提升决策连贯性
- 仅用500次评估即达90%单属性优化成功率(基线1.5倍)
- 适合资源有限的药物研发团队,尤其擅长复杂多目标优化
在药物发现中,分子优化旨在迭代改进先导化合物,以提升分子性能并保持与原始分子的结构相似性。然而每次模型评估成本高昂,样本效率成为关键挑战。传统试错方法耗时且依赖外部知识的方法易重复常见模板,在困难目标上表现不佳。核心缺失在于可支撑长期决策的可复用记忆机制。为此,我们提出MolMem(分子优化记忆增强框架),一种多轮代理强化学习架构,配备双记忆系统:静态示例记忆用于冷启动引导,演化技能记忆将成功优化路径提炼为可复用策略。基于此记忆增强设计,我们采用密集步骤奖励训练策略,将昂贵的完整优化过程转化为长期知识,显著提升后续优化效率。大量实验表明,MolMem在单属性任务中成功率高达90%(比最佳基线高出1.5倍),在多属性任务中达52%,仅需500次评估。代码已开源。
原文摘要 · Abstract (English)
In drug discovery, molecular optimization aims to iteratively refine a lead compound to improve molecular properties while preserving structural similarity to the original molecule. However, each oracle evaluation is expensive, making sample efficiency a key challenge for existing methods under a limited oracle budget. Trial-and-error approaches require many oracle calls, while methods that leverage external knowledge tend to reuse familiar templates and struggle on challenging objectives. A key missing piece is long-term memory that can ground decisions and provide reusable insights for future optimizations. To address this, we present MolMem (\textbf{Mol}ecular optimization with \textbf{Mem}ory), a multi-turn agentic reinforcement learning (RL) framework with a dual-memory system. Specifically, MolMem uses Static Exemplar Memory to retrieve relevant exemplars for cold-start grounding, and Evolving Skill Memory to distill successful trajectories into reusable strategies. Built on this memory-augmented formulation, we train the policy with dense step-wise rewards, turning costly rollouts into long-term knowledge that improves future optimization. Extensive experiments show that MolMem achieves 90\% success on single-property tasks (1.5$\times$ over the best baseline) and 52\% on multi-property tasks using only 500 oracle calls. Our code is available at https://github.com/REAL-Lab-NU/MolMem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。