arXiv:2607.16646cs.SEcs.AI2026-07被引 1

用数学验证方法检测大模型生成的优化模型错误,零误报且可定位缺陷。

Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits

  • 基于对偶与敏感性分析设计可验证的测试集,每项测试都严格对应问题描述中的角色。
  • 在326个真实模型中0%误报,检出56.1%核心建模错误,盲区预测准确。
  • 适合需要高可靠性的优化模型部署者,尤其适用于无标注参考场景。

大语言模型将自然语言描述转化为可求解的优化模型,但会无声失败:生成模型虽能运行却可能编码了错误问题,而传统评估依赖未提供的标签答案。本文提出基于反证的验证方法——每个数值量在问题描述中均有明确定义角色(如容量、需求、单位成本),正确模型必须按这些角色响应变化。基于对偶理论和敏感性分析,构建一组独立可靠的求解器测试,任何违反即证明模型错误,且设计上误报率为零。我们刻画了测试无法发现的错误类型,给出各类典型错误被完全检测的条件,并证明:具有调优阈值的扰动测试无法同时做到无误报与非平凡检测。在326个真实模型、一个合成族及四个公开基准(含两个生成器)上,该测试电池对忠实模型0%误报,而阈值测试误报54.9%;检出56.1%的核心建模错误,70.0%在认证前提下的错误,以及40.4%价值评分无法察觉的错误,且重现了预测的可检测模式及其盲区。每次警告均附带机器可验证的缺陷定位证书,完整审计仅需约25毫秒级求解器调用。对偶与敏感性分析为无需标签的优化模型提供严谨审计,补足现有评估体系。

原文摘要 · Abstract (English)

Large language models now translate natural-language descriptions of decision problems into solver-ready optimization models, and they fail silently. A generated model often runs and still encodes the wrong problem, while standard evaluation compares optimal values against labeled answers that deployment does not provide. How to certify such a model without any reference is the question this paper addresses. We develop falsification-based verification. Every numeric quantity in a problem description plays a role that the text itself states, such as a capacity, a requirement, or a unit cost, and any correct model must respond to changes in these quantities as the stated roles dictate. From duality and sensitivity analysis we derive a battery of solver-based tests that are individually sound, so a violation certifies a faulty model and the false-positive rate is zero by design. We characterize the errors that no test of this kind can see, give conditions under which each canonical error class is detected with certainty, and prove that perturbation testers with tuned thresholds cannot be simultaneously sound and nontrivial. Across 326 ground-truth models, a synthetic family, and four public benchmarks with two generators, the battery flags 0.0% of faithful models while a threshold tester flags 54.9%; it detects 56.1% of core formulation errors, 70.0% under certified preconditions, and 40.4% of the errors that value-based scoring provably cannot see, and it reproduces the predicted detectability pattern including its blind spots. Every flag carries a machine-checkable certificate that localizes the defect, and a full audit costs about 25 millisecond-scale solver calls per model. Classical sensitivity analysis and duality thus offer a rigorous, label-free audit that complements existing evaluation of AI-generated optimization models.

模型验证优化建模大模型审计反证法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。