arXiv:2605.08941cs.AI2026-05被引 2

测试AI代理在分子模拟中的自主能力,发现其表现远低于预期。

MDGYM: Benchmarking AI Agents on Molecular Simulations

论文配图:MDGYM: Benchmarking AI Agents on Molecular Simulations
图 1 · 摘自论文原文
  • 构建169个专家标注的分子动力学任务,分三个难度层级。
  • 最强代理仅解决21%简单任务,高难度下成功率不足10%。
  • 失败主因是物理不稳定性或虚构数据,非代码错误。

AI驱动科学发现的潜力取决于其能否自主设计并执行计算工作流。分子动力学(MD)模拟是检验这一能力的理想场景:需将物理直觉转化为语法与语义正确的脚本,合理设置初始与边界条件,诊断数值不稳定的轨迹,并根据已知物理规律解读输出。我们提出MDGYM基准,包含169个由专家精心筛选的模拟任务,覆盖LAMMPS和GROMACS两大主流软件,分为三个递增难度等级。评估了三种代理框架(Claude Code、Codex、OpenHands)与四种LLM的表现,结果均不理想:即使最强代理也仅能完成21%的简单任务,高难度下成功率低于10%。轨迹分析揭示典型失败模式——代理虽能调用模拟工具,但产生物理不稳定的构型,伪造数值结果而未实际运行计算,或在遇到模拟特定错误时过早放弃。这些失败模式与通用软件工程基准显著不同,表明流畅的代码生成无法迁移至基于物理的推理。

原文摘要 · Abstract (English)

The promise of AI-driven scientific discovery hinges on whether AI agents can autonomously design and execute the computational workflows that underpin modern science. Molecular dynamics (MD) simulation presents a natural test bed to stress-test this claim; it requires translating physical intuition into syntactically and semantically correct input scripts, reasoning about initial and boundary conditions, diagnosing numerically unstable trajectories, and interpreting outputs against known physical behavior and laws. We introduce MDGYM, a benchmark of 169 expert-curated MD simulations spanning LAMMPS and GROMACS, two widely used MD packages, across three increasing difficulty levels. We evaluate three agentic frameworks -- Claude Code, Codex, and OpenHands -- with four LLMs, and find that all perform poorly: even the strongest agent solves only 21\% of easy-level tasks, with less than 10\% at higher difficulties. Trajectory analysis reveals a characteristic pattern of failure -- agents successfully invoke simulation machinery but produce physically unstable configurations, fabricate numerical outputs without executing the underlying computation, or abandon tasks prematurely rather than iterating through simulation-specific errors. These failure modes are qualitatively distinct from those observed in general software engineering benchmarks, indicating that fluent code generation does not transfer to grounded physical reasoning.

分子模拟AI代理基准测试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。