arXiv:2601.19924cs.CLcs.AI2026-01被引 3

测试大模型在优化建模中的极限,发现当前方法难以处理复杂约束。

OPT-Engine: Benchmarking the Limits of LLMs in Optimization Modeling via Complexity Scaling

  • 构建可扩展的基准框架,系统化测试大模型在10类运筹问题中的表现。
  • 纯文本推理在复杂任务中鲁棒性差,外部工具仍无法满足全局约束。
  • 当前最优范式下,自动约束生成是主要瓶颈,适合优化与AI交叉研究者。

我们研究大语言模型(LLMs)在优化建模中的能力与可扩展性,该领域需要结构化推理与精确公式表达。为此,我们提出OPT-ENGINE,一个可扩展、复杂度可量化控制的基准框架。它涵盖10个经典运筹学问题,从线性规划系统性扩展至混合整数规划,提供结构化环境以探究自动化问题建模与求解的极限。利用OPT-Engine,我们回答三个关键研究问题:首先,考察仅通过经典思维链(PTR)进行纯文本推理是否能高效解决优化任务,发现随着任务复杂度提升,PTR存在严重鲁棒性缺口;其次,考察集成外部计算工具能否缓解其算术弱点,结果表明此类工具虽有助于局部计算,但仍无法遵守全局优化约束;最后,我们指出,在当前最先进范式下,求解器集成推理(SIR)中自动化约束生成是主要瓶颈。这些发现明确了现有范式的局限性,并为下一代优化建模大模型的发展提供了结构化路线图。代码与数据已公开(https://github.com/Cardinal-Operations/OPTEngine)。

原文摘要 · Abstract (English)

We investigate the capabilities and scalability of Large Language Models (LLMs) in optimization modeling, a domain requiring structured reasoning and precise formulation. To this end, we introduce OPT-ENGINE, an extensible benchmark framework with quantifiable and controllable complexity. OPT-ENGINE spans ten canonical Operations Research problems, systematically scaling from Linear Programming to Mixed-Integer Programming, providing a structured environment to probe the limits of automated problem formulation and solving. Utilizing OPT-Engine, we address three pivotal research questions. First, we examine whether Pure-Text Reasoning (PTR) via classical Chain-of-Thought can efficiently tackle optimization tasks, finding that PTR suffers from a critical robustness gap as task complexity increases. Second, we examine whether integrating external computational tools can mitigate PTR's arithmetic weaknesses and improve performance. Our results indicate that while such tools help with local calculations, they still fail to adhere to global optimization constraints. Finally, we pinpoint that for the current SOTA paradigm, Solver-integrated Reasoning (SIR), the automated formulation of constraints represents the primary bottleneck. These findings clarify the limitations of current paradigms and provide a structured roadmap for developing next-generation LLMs for optimization modeling. We release our code and data to facilitate future research (https://github.com/Cardinal-Operations/OPTEngine).

大模型优化建模推理瓶颈运筹学

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。