用多智能体系统检测大模型优化建模中的幻觉问题
OptArgus: A Multi-Agent System to Detect Hallucinations in LLM-based Optimization Modeling
- 设计多智能体架构，分角色审查问题描述、符号模型和求解代码
- 在6292个真实生成的模型中，误报率更低，错误定位更准
- 首次建立优化建模幻觉细粒度分类体系，适合可信建模研究者
大型语言模型（LLMs）被广泛用于将自然语言优化问题转换为数学公式和求解代码，但仅匹配参考目标值无法可靠检验正确性：某些幻觉可能在数值上一致，却改变了底层优化语义。本文提出‘优化建模幻觉检测’任务，即对问题描述、符号模型和求解实现进行结构一致性审计。我们构建了首个专为优化建模设计的细粒度幻觉分类体系，涵盖目标、变量、约束和实现四类错误。基于此，开发了多智能体检测系统OptArgus，包含指挥者路由、专业审计员和证据整合模块。为评估该系统，引入三部分基准测试套件：484个干净样本、1266个可控注入错误样本、6292个自然生成的LLM输出。相比单智能体基线，OptArgus在干净样本上误报更少，在单错误案例中定位更准确，在真实生成模型上检测能力更强。这些成果使优化建模幻觉检测成为可实证的问题，并表明基于分类体系的模块化审计是提升建模可靠性的重要路径。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used to translate natural-language optimization problems into mathematical formulations and solver code, but matching the reference objective value is not a reliable test of correctness: an artifact may agree numerically while still changing the underlying optimization semantics. We formulate this issue as \emph{optimization-modeling hallucination detection}, namely structural consistency auditing over the problem description, symbolic model, and solver implementation. We develop, to our knowledge, the first fine-grained hallucination taxonomy specifically for optimization modeling, spanning objective, variable, constraint, and implementation failures. We use this taxonomy to design OptArgus, a multi-agent detector with conductor routing, specialist auditors, and evidence consolidation. To evaluate this setting, we introduce a three-part benchmark suite with $484$ clean artifacts, $1266$ controlled injected artifacts, and $6292$ natural LLM-generated artifacts. Against a matched single-agent baseline, OptArgus produces fewer false alarms on clean artifacts, more accurate top-ranked localization on controlled single-error cases, and stronger detection on natural model outputs. Together, these contributions turn optimization-modeling hallucination detection into a concrete empirical problem and suggest that modular, taxonomy-grounded auditing is a practical route to more reliable optimization modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。