arXiv:2410.13213cs.AIcs.LG2024-10ICLR被引 124

用大模型自动理解并求解各类优化问题,准确率提升11%。

LLMOPT: Learning to Define and Solve General Optimization Problems from Scratch

  • 通过五元组形式化框架统一建模不同优化问题
  • 多指令微调提升公式构建与代码生成精度
  • 自校正机制防止大模型幻觉,保障求解可靠性

优化问题广泛存在于医疗、环境、能源、制造等多个领域。将自然语言描述的优化问题进行形式化并求解,通常需要高度专业的人工干预,限制了优化决策的普及应用。为实现自动化问题建模与求解,基于大语言模型(LLM)的方法成为可能,但现有方法在优化问题泛化能力上仍有限。本文提出统一学习框架LLMOPT,从自然语言描述出发,利用预训练大模型构建五元组形式化表达,作为通用建模基础。通过多指令微调提升问题形式化与求解代码生成的准确性与泛化性。为进一步抑制大模型幻觉(如为避免执行错误牺牲求解精度),引入模型对齐与自校正机制。在涵盖约20个领域的六个真实数据集上评估,结果表明,LLMOPT可有效建模线性/非线性规划、混合整数规划、组合优化等多种问题类型,相比当前最优方法平均求解准确率提升11.08%。代码已开源。

原文摘要 · Abstract (English)

Optimization problems are prevalent across various scenarios. Formulating and then solving optimization problems described by natural language often requires highly specialized human expertise, which could block the widespread application of optimization-based decision making. To automate problem formulation and solving, leveraging large language models (LLMs) has emerged as a potential way. However, this kind of approach suffers from the issue of optimization generalization. Namely, the accuracy of most current LLM-based methods and the generality of optimization problem types that they can model are still limited. In this paper, we propose a unified learning-based framework called LLMOPT to boost optimization generalization. Starting from the natural language descriptions of optimization problems and a pre-trained LLM, LLMOPT constructs the introduced five-element formulation as a universal model for learning to define diverse optimization problem types. Then, LLMOPT employs the multi-instruction tuning to enhance both problem formalization and solver code generation accuracy and generality. After that, to prevent hallucinations in LLMs, such as sacrificing solving accuracy to avoid execution errors, the model alignment and self-correction mechanism are adopted in LLMOPT. We evaluate the optimization generalization ability of LLMOPT and compared methods across six real-world datasets covering roughly 20 fields such as health, environment, energy and manufacturing, etc. Extensive experiment results show that LLMOPT is able to model various optimization problem types such as linear/nonlinear programming, mixed integer programming, and combinatorial optimization, and achieves a notable 11.08% average solving accuracy improvement compared with the state-of-the-art methods. The code is available at https://github.com/caigaojiang/LLMOPT.

优化求解大模型自动化建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。