arXiv:2503.01675cs.LGcs.CL2025-03被引 10

用70亿参数小模型把自然语言转成反应网络模拟模型,效果接近大模型。

Using (Not-so) Large Language Models to Generate Simulation Models in a Formal DSL: A Study on Reaction Networks

  • 用合成数据微调70亿参数的Mistral模型,实现自然语言到形式化反应网络的转换。
  • 在84.5%的案例中能准确恢复真实模拟模型,用户测试显示适用于单次生成与交互建模。
  • 开源小模型成本低、可自托管,适合资源有限的研究者和特定领域应用。

形式化语言是建模与仿真的核心,能将知识浓缩为可自动执行、解析和分析的仿真模型。然而,人类最易表达模型的方式是自然语言,而计算机难以直接理解。本文评估大型语言模型(LLM)将自然语言转化为仿真模型的潜力。现有研究仅使用未微调的超大规模商业模型(如GPT)。为此,我们展示如何对一个开源权重、70亿参数的Mistral模型进行微调,将其用于将自然语言描述转换为领域特定语言(DSL)中的反应网络模型,提供一种可自托管、计算高效且内存高效的替代方案。为此,我们开发了合成数据生成器,作为微调与评估的基础。定量评估表明,微调后的Mistral模型在高达84.5%的案例中可恢复真实模型。此外,小规模用户研究表明该模型在一次性生成及跨领域交互建模中具有实际潜力。尽管前景良好,当前版本的小型微调模型仍无法媲美大型模型。我们得出结论:需更高质训练数据,未来小型开源模型有望带来新机遇。

原文摘要 · Abstract (English)

Formal languages are an integral part of modeling and simulation. They allow the distillation of knowledge into concise simulation models amenable to automatic execution, interpretation, and analysis. However, the arguably most humanly accessible means of expressing models is through natural language, which is not easily interpretable by computers. Here, we evaluate how a Large Language Model (LLM) might be used for formalizing natural language into simulation models. Existing studies only explored using very large LLMs, like the commercial GPT models, without fine-tuning model weights. To close this gap, we show how an open-weights, 7B-parameter Mistral model can be fine-tuned to translate natural language descriptions to reaction network models in a domain-specific language, offering a self-hostable, compute-efficient, and memory efficient alternative. To this end, we develop a synthetic data generator to serve as the basis for fine-tuning and evaluation. Our quantitative evaluation shows that our fine-tuned Mistral model can recover the ground truth simulation model in up to 84.5% of cases. In addition, our small-scale user study demonstrates the model's practical potential for one-time generation as well as interactive modeling in various domains. While promising, in its current form, the fine-tuned small LLM cannot catch up with large LLMs. We conclude that higher-quality training data are required, and expect future small and open-source LLMs to offer new opportunities.

大模型形式化建模反应网络小模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。