大模型能复现复杂仿真模型吗？实验发现仅部分模型可稳定生成可运行代码。
Can Large Language Models Implement Agent-Based Models? An ODD-based Replication Study
- 用标准规范指令让大模型生成代理模型代码，测试可复现性
- GPT-4.1表现最佳，生成代码行为与基准一致且效率高
- 代码可运行不等于科学可用，需额外验证机制
大型语言模型（LLMs）现在能从文本描述生成非平凡的可执行代码，引发一个重要问题：大模型能否可靠地从标准化规范中实现代理模型，以支持复现、验证和确认？我们通过一个受控的ODD-to-code任务评估了17个当代大模型，使用PPHPC捕食者-猎物模型作为完整指定的参考。生成的Python实现通过分阶段可执行性检查、与已验证的NetLogo基线进行模型无关的统计对比，以及运行时效率和可维护性的定量度量进行评估。结果显示，行为上忠实的实现是可能的但并非必然，且仅可执行性不足以支持科学应用。GPT-4.1始终生成统计有效且高效的实现，Claude 3.7 Sonnet表现良好但可靠性较低。总体而言，研究明确了大模型作为模型工程工具的潜力与当前局限，对可复现的代理建模和生态模拟具有启示意义。
原文摘要 · Abstract (English)
Large language models (LLMs) can now synthesize non-trivial executable code from textual descriptions, raising an important question: can LLMs reliably implement agent-based models from standardized specifications in a way that supports replication, verification, and validation? We address this question by evaluating 17 contemporary LLMs on a controlled ODD-to-code translation task, using the PPHPC predator-prey model as a fully specified reference. Generated Python implementations are assessed through staged executability checks, model-independent statistical comparison against a validated NetLogo baseline, and quantitative measures of runtime efficiency and maintainability. Results show that behaviorally faithful implementations are achievable but not guaranteed, and that executability alone is insufficient for scientific use. GPT-4.1 consistently produces statistically valid and efficient implementations, with Claude 3.7 Sonnet performing well but less reliably. Overall, the findings clarify both the promise and current limitations of LLMs as model engineering tools, with implications for reproducible agent-based and ecological modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。