arXiv:2602.10450cs.LGcs.AI2026-02中稿 · ICML被引 5

构建工业级优化建模基准,让大模型真实评估建模能力

Constructing Industrial-Scale Optimization Modeling Benchmark

  • 从真实混合整数规划实例反向生成带结构的自然语言描述
  • 223个实例含10^3至10^6量级变量与约束,覆盖工业规模
  • 专为测试大模型真实建模能力设计,适合评估工业级应用

优化建模支撑物流、制造、能源与金融等领域的决策,但将自然语言需求转化为正确优化公式和可执行代码仍需大量人力。尽管已有大语言模型尝试解决该问题,现有评估仍依赖小规模或合成基准,掩盖了工业级问题中变量与约束达10^3至10^6(或更多)的真实难度。核心瓶颈在于缺乏将自然语言规范与真实优化模型对应的基准。为此,我们提出MIPLIB-NL,通过结构感知的逆向构建方法,从MIPLIB 2017中的真实混合整数线性规划实例生成。该流程(i)从扁平化求解器公式中恢复紧凑可复用的模型结构;(ii)在统一模型-数据分离格式下,逆向生成与该结构紧密关联的自然语言描述;(iii)通过专家评审与人-大模型交互式迭代语义验证,结合独立重建检查。最终获得223个一一对应的重建实例,在保持原实例数学内容的同时,支持真实的自然语言到优化建模评估。实验表明,现有在传统基准上表现优异的系统在MIPLIB-NL上性能显著下降,暴露出小规模基准无法揭示的失效模式。

原文摘要 · Abstract (English)

Optimization modeling underpins decision-making in logistics, manufacturing, energy, and finance, yet translating natural-language requirements into correct optimization formulations and solver-executable code remains labor-intensive. Although large language models (LLMs) have been explored for this task, evaluation is still dominated by toy-sized or synthetic benchmarks, masking the difficulty of industrial problems with $10^{3}$--$10^{6}$ (or more) variables and constraints. A key bottleneck is the lack of benchmarks that align natural-language specifications with reference formulations/solver code grounded in real optimization models. To fill in this gap, we introduce MIPLIB-NL, built via a structure-aware reverse construction methodology from real mixed-integer linear programs in MIPLIB~2017. Our pipeline (i) recovers compact, reusable model structure from flat solver formulations, (ii) reverse-generates natural-language specifications explicitly tied to this recovered structure under a unified model--data separation format, and (iii) performs iterative semantic validation through expert review and human--LLM interaction with independent reconstruction checks. This yields 223 one-to-one reconstructions that preserve the mathematical content of the original instances while enabling realistic natural-language-to-optimization evaluation. Experiments show substantial performance degradation on MIPLIB-NL for systems that perform strongly on existing benchmarks, exposing failure modes invisible at toy scale.

优化建模工业级大模型评估基准构建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。