提升大模型数学建模推理能力，通过强化学习优化思维树搜索。
BPP-Search: Enhancing Tree of Thought Reasoning for Mathematical Modeling Problem Solving
- 将强化学习融入思维树结构，结合束搜索与偏好算法
- 在多个数据集上准确率显著超越现有方法，推理速度更快
- 开源完整标注的建模过程数据集，适合研究者复现与训练
大语言模型具备强大的推理能力，有望将自然语言问题转化为数学模型。然而，现有开源运筹学数据集缺乏变量定义等建模过程的详细标注，仅提供目标值，限制了强化学习的应用。为此，我们发布了包含完整建模过程标注的StructuredOR数据集。进一步提出BPP-Search算法，将强化学习集成到树状思维结构中，结合束搜索、过程奖励模型和成对偏好算法，实现高效探索，避免全量搜索的同时提升准确性。在StructuredOR、NL4OPT和MAMO-ComplexLP数据集上的大量实验表明，BPP-Search显著优于当前最优方法。在基于树的推理中，该方法在准确率和效率方面表现优异，能更快找到正确解。StructuredOR数据集已公开于Huggingface：https://huggingface.co/datasets/LLM4OR/StructuredOR 及 GitHub：https://github.com/LLM4OR/StructuredOR。
原文摘要 · Abstract (English)
LLMs exhibit advanced reasoning capabilities, offering the potential to transform natural language questions into mathematical models. However, existing open-source datasets in operations research domain lack detailed annotations of the modeling process, such as variable definitions, focusing solely on objective values, which hinders reinforcement learning applications. To address this, we release the StructuredOR dataset, annotated with comprehensive labels that capture the complete mathematical modeling process. We further propose BPP-Search, an algorithm that integrates reinforcement learning into a tree-of-thought structure using Beam search, a Process reward model, and a pairwise Preference algorithm. This approach enables efficient exploration of tree structures, avoiding exhaustive search while improving accuracy. Extensive experiments on StructuredOR, NL4OPT, and MAMO-ComplexLP datasets show that BPP-Search significantly outperforms state-of-the-art methods. In tree-based reasoning, BPP-Search excels in accuracy and efficiency, enabling faster retrieval of correct solutions. The StructuredOR dataset is available on Huggingface https://huggingface.co/datasets/LLM4OR/StructuredOR and GitHub https://github.com/LLM4OR/StructuredOR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。