用少量数据训练的AI,能自动把问题描述变成可解的优化模型。
OR-R1: Automating Modeling and Solving of Operations Research Optimization Problem via Test-Time Reinforcement Learning
- 先用少量标注数据学基础思路,再用无标注数据通过测试时优化提升能力。
- 仅用1/10的合成数据,解题准确率达67.7%,比前人高4.2%。
- 适合想快速部署优化求解但缺乏专家经验的工业用户。
优化建模与求解是运筹学在现实决策中的核心环节,但将自然语言问题描述转化为正式模型和求解代码仍高度依赖专业技能。尽管大语言模型带来自动化新可能,现有方法泛化能力弱、数据效率低,多需大量标注或合成数据,导致成本高、难扩展。本文提出OR-R1,一种高效数据训练框架,实现自动化优化建模与求解。该框架首先通过监督微调(SFT)在有限标注数据下学习关键推理模式;其次引入测试时组相对策略优化(TGRPO),增强模型能力与一致性。两阶段设计使OR-R1能同时利用稀缺标注数据与丰富无标注数据。实验显示,其平均求解准确率达67.7%,仅需前人方法如ORLM所需合成数据的1/10,且准确率最高提升4.2%;仅用100个合成样本即超越ORLM超2.4%。TGRPO额外带来3.1%–6.4%准确率提升,使单次尝试(Pass@1)与八次尝试(Pass@8)性能差距从13%缩小至7%。跨多样真实场景评估表明,OR-R1为运筹优化提供了鲁棒、可扩展且低成本的自动化解决方案,显著降低工业应用中的专业知识与数据门槛。
原文摘要 · Abstract (English)
Optimization modeling and solving are fundamental to the application of Operations Research (OR) in real-world decision making, yet the process of translating natural language problem descriptions into formal models and solver code remains highly expertise intensive. While recent advances in large language models (LLMs) have opened new opportunities for automation, the generalization ability and data efficiency of existing LLM-based methods are still limited, asmost require vast amounts of annotated or synthetic data, resulting in high costs and scalability barriers. In this work, we present OR-R1, a data-efficient training framework for automated optimization modeling and solving. OR-R1 first employs supervised fine-tuning (SFT) to help the model acquire the essential reasoning patterns for problem formulation and code generation from limited labeled data. In addition, it improves the capability and consistency through Test-Time Group Relative Policy Optimization (TGRPO). This two-stage design enables OR-R1 to leverage both scarce labeled and abundant unlabeled data for effective learning. Experiments show that OR-R1 achieves state-of-the-art performance with an average solving accuracy of $67.7\%$, using only $1/10$ the synthetic data required by prior methods such as ORLM, exceeding ORLM's solving accuracy by up to $4.2\%$. Remarkably, OR-R1 outperforms ORLM by over $2.4\%$ with just $100$ synthetic samples. Furthermore, TGRPO contributes an additional $3.1\%-6.4\%$ improvement in accuracy, significantly narrowing the gap between single-attempt (Pass@1) and multi-attempt (Pass@8) performance from $13\%$ to $7\%$. Extensive evaluations across diverse real-world benchmarks demonstrate that OR-R1 provides a robust, scalable, and cost-effective solution for automated OR optimization problem modeling and solving, lowering the expertise and data barriers for industrial OR applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。