arXiv:2510.22131cs.LGcs.AI2025-10NeurIPS被引 3

用探针分析神经组合优化模型的内部机制,揭示其决策逻辑。

Probing Neural Combinatorial Optimization Models

  • 设计新探针工具CS-Probing,分析模型系数与统计显著性。
  • 发现模型编码低层构建信息和高层决策知识。
  • 可指导改进模型泛化能力,适合研究者与工程师参考。

神经组合优化(NCO)模型性能优异,但其内部表示与决策逻辑仍属黑箱,阻碍学术研究与实际应用。本文首次系统探究NCO模型的表征,通过多种探针任务揭示其内部机制。提出新型探针工具系数显著性探针(CS-Probing),通过分析探针过程中的系数及其统计显著性,实现更深入的分析。大量实验表明,NCO模型既编码解构建所需的底层信息,也捕捉有助于决策的高层知识。借助CS-Probing,发现主流模型对学习表征施加不同归纳偏置,获得直接支持模型泛化的证据,并识别出与特定知识相关的关键嵌入维度。这些发现可转化为实践,仅需少量代码修改即可提升模型泛化能力。本工作是首个系统解析黑箱NCO模型的研究,展示了探针技术在分析内部机制中的潜力,为NCO社区提供重要洞见。源代码公开。

原文摘要 · Abstract (English)

Neural combinatorial optimization (NCO) has achieved remarkable performance, yet its learned model representations and decision rationale remain a black box. This impedes both academic research and practical deployment, since researchers and stakeholders require deeper insights into NCO models. In this paper, we take the first critical step towards interpreting NCO models by investigating their representations through various probing tasks. Moreover, we introduce a novel probing tool named Coefficient Significance Probing (CS-Probing) to enable deeper analysis of NCO representations by examining the coefficients and statistical significance during probing. Extensive experiments and analysis reveal that NCO models encode low-level information essential for solution construction, while capturing high-level knowledge to facilitate better decisions. Using CS-Probing, we find that prevalent NCO models impose varying inductive biases on their learned representations, uncover direct evidence related to model generalization, and identify key embedding dimensions associated with specific knowledge. These insights can be potentially translated into practice, for example, with minor code modifications, we improve the generalization of the analyzed model. Our work represents a first systematic attempt to interpret black-box NCO models, showcasing probing as a promising tool for analyzing their internal mechanisms and revealing insights for the NCO community. The source code is publicly available.

组合优化模型解释探针分析机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。