新模型无需求解器即可学习复杂推理问题的约束与目标,训练快且精度高。
Scaling Neuro-symbolic Problem Solving: Solver-Free Learning of Constraints and Objectives
- 用可微分神经符号架构联合学习约束与目标函数
- 在三类数独任务上训练时间仅为其他方法的几分之一
- 适合需要高效学习复杂优化问题的科研与工业场景
在融合离散推理与神经网络的探索中,研究者日益关注能从自然输入中学习求解离散推理或优化问题的神经架构,而大型语言模型在此类任务上表现不佳。本文提出一种可微分神经符号架构及专用损失函数,用于学习求解NP-hard推理问题。该概率性损失函数支持同时学习约束和目标,从而构建出可审查并添加额外约束的完整模型。通过将组合求解器移出训练循环,该架构实现可扩展训练;而精确推理则保证了最高精度。实验表明,该方法能高效从自然输入中学习求解NP-hard推理问题。在三种数独变体(符号、视觉、多解)上,其训练时间仅为其他混合方法的几分之一;在视觉最小割/最大割任务中,优化遗憾优于专门设计的决策导向学习损失;此外,还能高效学习蛋白质设计这一大规模真实世界问题的能量优化形式。
原文摘要 · Abstract (English)
In the ongoing quest for hybridizing discrete reasoning with neural nets, there is an increasing interest in neural architectures that can learn how to solve discrete reasoning or optimization problems from natural inputs, a task that Large Language Models seem to struggle with. Objectives: We introduce a differentiable neuro-symbolic architecture and a loss function dedicated to learning how to solve NP-hard reasoning problems. Methods: Our new probabilistic loss allows for learning both the constraints and the objective, thus delivering a complete model that can be scrutinized and completed with side constraints. By pushing the combinatorial solver out of the training loop, our architecture also offers scalable training while exact inference gives access to maximum accuracy. Results: We empirically show that it can efficiently learn how to solve NP-hard reasoning problems from natural inputs. On three variants of the Sudoku benchmark -- symbolic, visual, and many-solution --, our approach requires a fraction of training time of other hybrid methods. On a visual Min-Cut/Max-cut task, it optimizes the regret better than a Decision-Focused-Learning regret-dedicated loss. Finally, it efficiently learns the energy optimization formulation of the large real-world problem of designing proteins.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。