构建工业级优化建模基准,让大模型真实评估建模能力
Constructing Industrial-Scale Optimization Modeling Benchmark
- 从真实混合整数规划实例反向生成带结构的自然语言描述
- 223个实例含10^3至10^6量级变量与约束,覆盖工业规模
- 专为测试大模型真实建模能力设计,适合评估工业级应用
优化建模支撑物流、制造、能源与金融等领域的决策,但将自然语言需求转化为正确优化公式和可执行代码仍需大量人力。尽管已有大语言模型尝试解决该问题,现有评估仍依赖小规模或合成基准,掩盖了工业级问题中变量与约束达10^3至10^6(或更多)的真实难度。核心瓶颈在于缺乏将自然语言规范与真实优化模型对应的基准。为此,我们提出MIPLIB-NL,通过结构感知的逆向构建方法,从MIPLIB 2017中的真实混合整数线性规划实例生成。该流程(i)从扁平化求解器公式中恢复紧凑可复用的模型结构;(ii)在统一模型-数据分离格式下,逆向生成与该结构紧密关联的自然语言描述;(iii)通过专家评审与人-大模型交互式迭代语义验证,结合独立重建检查。最终获得223个一一对应的重建实例,在保持原实例数学内容的同时,支持真实的自然语言到优化建模评估。实验表明,现有在传统基准上表现优异的系统在MIPLIB-NL上性能显著下降,暴露出小规模基准无法揭示的失效模式。
原文摘要 · Abstract (English)
Optimization modeling underpins decision-making in logistics, manufacturing, energy, and finance, yet translating natural-language requirements into correct optimization formulations and solver-executable code remains labor-intensive. Although large language models (LLMs) have been explored for this task, evaluation is still dominated by toy-sized or synthetic benchmarks, masking the difficulty of industrial problems with $10^{3}$--$10^{6}$ (or more) variables and constraints. A key bottleneck is the lack of benchmarks that align natural-language specifications with reference formulations/solver code grounded in real optimization models. To fill in this gap, we introduce MIPLIB-NL, built via a structure-aware reverse construction methodology from real mixed-integer linear programs in MIPLIB~2017. Our pipeline (i) recovers compact, reusable model structure from flat solver formulations, (ii) reverse-generates natural-language specifications explicitly tied to this recovered structure under a unified model--data separation format, and (iii) performs iterative semantic validation through expert review and human--LLM interaction with independent reconstruction checks. This yields 223 one-to-one reconstructions that preserve the mathematical content of the original instances while enabling realistic natural-language-to-optimization evaluation. Experiments show substantial performance degradation on MIPLIB-NL for systems that perform strongly on existing benchmarks, exposing failure modes invisible at toy scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。