构建首个多模态优化建模基准，测试模型从图文混合信息生成可执行代码的能力。
MM-OptBench: A Solver-Grounded Benchmark for Multimodal Optimization Modeling
- 提出多模态优化建模新任务，需从文本和图表中提取数据并生成数学模型与求解代码。
- 建立780个经求解器验证的实例，覆盖6类26子类问题，难度分三级，最佳模型准确率仅52.1%。
- 首次揭示模型在图文信息提取与公式转化两环节均存在系统性错误，适合决策智能研究者使用。
优化建模将实际决策问题转化为数学优化模型及可执行求解器代码。尽管语言模型被广泛用于生成优化形式和求解代码，现有基准几乎全为纯文本，忽略了实践中大量依赖图表、表格、地图、日程等视觉信息的场景。本文提出多模态优化建模新范式：模型需从文本与视觉混合描述中构建数学公式与可运行求解代码。为此，我们设计了基于求解器验证的框架，统一生成结构化实例、验证结果，并构建模型输入与参考答案。基于此框架，我们构建了MM-OptBench基准，包含780个经精确求解器验证的实例，覆盖6类优化家族、26个子类及3种结构难度等级。我们评估9个多模态大模型（6个通用+3个数学专用），涵盖聚合、类别、难度与失败模式分析。结果显示该任务仍远未解决：最佳两模型通过率分别为52.1%和51.3%，六款通用模型在易例上平均43.4%，难例仅15.9%；三款数学专用模型全错（0/780）。错误主要源于从图文中提取数据或转换为正确求解代码阶段。该基准为求解器驱动的决策型多模态智能提供评测平台。
原文摘要 · Abstract (English)
Optimization modeling translates real decision-making problems into mathematical optimization models and solver-executable implementations. Although language models are increasingly used to generate optimization formulations and solver code, existing benchmarks are almost entirely text-only. This omits many optimization-modeling tasks that arise in operational practice, where requirements are described in text but instance information is conveyed through visual artifacts such as tables, graphs, maps, schedules, and dashboards. We introduce multimodal optimization modeling, a benchmark setting in which models must construct both a mathematical formulation and executable solver code from a text-and-visual problem specification. To evaluate this setting, we develop a solver-grounded framework that generates structured optimization instances, verifies each with an exact solver, and builds both the model-facing inputs and hidden reference files from the same verified source. We instantiate the framework as MM-OptBench, a benchmark of 780 solver-verified instances spanning 6 optimization families, 26 subcategories, and 3 structural difficulty levels. We evaluate 9 multimodal large language models (MLLMs), including 6 frontier general-purpose models and 3 math-specialized models, with aggregate, family-level, difficulty-level, and failure-mode analyses. The results show that the task remains far from solved: the best two models reach 52.1% and 51.3% pass@1, while on average across the six general-purpose MLLMs, pass@1 is 43.4% on easy instances and 15.9% on hard instances. All three math-specialized MLLMs solve 0/780 instances. Failure attribution shows that errors arise both when extracting instance data from text and visuals and when turning extracted data into solver-correct formulations and code. MM-OptBench provides a testbed for solver-grounded, decision-oriented multimodal intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。