arXiv:2411.01679cs.LG2024-11ICML被引 60

用大模型自动把自然语言问题转为可求解的优化模型

Autoformulation of Mathematical Optimization Models Using LLMs

  • 结合大模型与蒙特卡洛树搜索,系统生成优化模型
  • 在多个基准上提升求解效率,性能显著优于基线
  • 适合需要快速建模的工程师或非专业用户

数学优化是运筹学、医疗等多个领域决策的基础,但将现实问题转化为优化模型仍需专业知识。本文研究自动化建模(autoformulation):从自然语言描述自动生成可被求解器使用的优化模型。识别出三大挑战:(1) 问题依赖的庞大假设空间,(2) 在不确定性下高效多样地探索该空间,(3) 评估建模结果是否正确匹配原始描述。为此,提出一种新方法,利用大语言模型(LLMs)与蒙特卡洛树搜索,借助优化建模的层次结构生成并系统探索可能的建模路径。为提高搜索效率,引入符号剪枝以消除等价路径,并使用基于LLM的部分模型评估来引导搜索。在线性与混合整数规划基准上的实证分析表明,该方法在两种技术——基于LLM的价值估计和符号剪枝——协同作用下表现显著提升。

原文摘要 · Abstract (English)

Mathematical optimization is fundamental to decision-making across diverse domains, from operations research to healthcare. Yet, translating real-world problems into optimization models remains a difficult task, often demanding specialized expertise. This paper approaches the problem of $\textit{autoformulation}$: the automated creation of solver-ready optimization models from natural language problem descriptions. We identify three core challenges of autoformulation: $\textit{(1)}$ the vast, problem-dependent hypothesis space, $\textit{(2)}$ efficient and diverse exploration of this space under uncertainty, and $\textit{(3)}$ evaluation of formulation correctness against problem description. To address these challenges, we present a novel method leveraging $\textit{Large Language Models}$ (LLMs) with $\textit{Monte-Carlo Tree Search}$, exploiting the hierarchical nature of optimization modeling to generate and systematically explore possible formulations. To enhance search efficiency, we introduce symbolic pruning to eliminate trivially equivalent search paths (branches), and employ LLM-based evaluation of partial formulations to guide search. Empirical analysis on linear and mixed-integer programming benchmarks demonstrates our method's effectiveness, with significant performance gains from both LLM-based value estimation and symbolic pruning techniques.

优化建模大模型应用自动推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。