通过相似表示共用标签，高效优化神经符号学习中的逻辑约束分类。
On Improving Neurosymbolic Learning by Exploiting the Representation Space
- 利用实例隐层表示相似性，智能剪枝满足逻辑公式的标签组合空间。
- 在16个复杂任务上，使Scallop、Dolphin等模型性能提升最高达53%。
- 可无缝集成到现有训练框架，适合处理带逻辑约束的弱监督分类问题。
本文研究神经符号设置下神经分类器的学习问题，其中输入实例的隐藏真标签需满足逻辑公式。学习过程首先计算满足公式的标签组合子集，再基于这些组合与分类器得分计算损失。难点在于标签组合空间可能指数级增长，导致学习困难。我们提出一种剪枝技术，利用相似隐表示的实例很可能共享相同标签的直觉。尽管该直觉广泛用于弱监督学习，但在逻辑约束下应用极具挑战。我们将剪枝过程建模为整数线性规划，剔除不一致标签组合的同时保留逻辑结构。所提方法CLIPPER与现有训练算法正交，可无缝集成。在16个复杂神经符号任务上，其使Scallop、Dolphin和ISED等先进引擎的性能分别提升最高48%、53%和8%，达到当前最优准确率。
原文摘要 · Abstract (English)
We study the problem of learning neural classifiers in a neurosymbolic setting where the hidden gold labels of input instances must satisfy a logical formula. Learning in this setting proceeds by first computing (a subset of) the possible combinations of labels that satisfy the formula and then computing a loss using those combinations and the classifiers' scores. One challenge is that the space of label combinations can grow exponentially, making learning difficult. We propose a technique that prunes this space by exploiting the intuition that instances with similar latent representations are likely to share the same label. While this intuition has been widely used in weakly supervised learning, its application in our setting is challenging due to label dependencies imposed by logical constraints. We formulate the pruning process as an integer linear program that discards inconsistent label combinations while respecting logical structure. Our approach, CLIPPER, is orthogonal to existing training algorithms and can be seamlessly integrated with them. Across 16 benchmarks over complex neurosymbolic tasks, we demonstrate that CLIPPER boosts the performance of state-of-the-art neurosymbolic engines like Scallop, Dolphin, and ISED by up to 48%, 53%, and 8%, leading to state-of-the-art accuracies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。