测试大模型生成线性规划对偶问题的能力，发现其表现远不如预期。
DualSchool: How Reliable are LLMs for Optimization Education?
- 构建DualSchool框架，系统生成并验证对偶转换实例
- 顶尖开源大模型在双变量实例中仍频繁出错
- 适合教育者与大模型推理系统开发者参考
本文针对优化教育中一个基础任务——生成线性规划的对偶问题，评估大语言模型（LLMs）的表现。尽管这些模型在网页规模数据上训练，掌握大量原始问题到对偶问题的转换实例，但实验表明，即使是最小的两变量问题，主流开源大模型也难以稳定生成正确对偶。为此，本文提出DualSchool框架，用于生成和验证对偶转换实例，其验证过程采用标准图编辑距离（Canonical Graph Edit Distance），显著优于现有方法在对偶转换任务中普遍存在的假阳性与假阴性问题。研究发现，尽管模型能准确复述转换规则，但在实际输出中错误频发，且在正确性判断、验证和错误分类等衍生任务上也表现不佳。该结果对教学实践和大型推理系统设计具有重要启示。
原文摘要 · Abstract (English)
Consider the following task taught in introductory optimization courses which addresses challenges articulated by the community at the intersection of (generative) AI and OR: generate the dual of a linear program. LLMs, being trained at web-scale, have the conversion process and many instances of Primal to Dual Conversion (P2DC) at their disposal. Students may thus reasonably expect that LLMs would perform well on the P2DC task. To assess this expectation, this paper introduces DualSchool, a comprehensive framework for generating and verifying P2DC instances. The verification procedure of DualSchool uses the Canonical Graph Edit Distance, going well beyond existing evaluation methods for optimization models, which exhibit many false positives and negatives when applied to P2DC. Experiments performed by DualSchool reveal interesting findings. Although LLMs can recite the conversion procedure accurately, state-of-the-art open LLMs fail to consistently produce correct duals. This finding holds even for the smallest two-variable instances and for derivative tasks, such as correctness, verification, and error classification. The paper also discusses the implications for educators, students, and the development of large reasoning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。