用命题逻辑测试神经网络泛化能力,发现标准模型难处理否定等复杂逻辑。
Propositional Logic for Probing Generalization in Neural Networks
- 设计基于命题逻辑的可控任务,用新数据集消除表面模式干扰。
- 三类模型在已知逻辑组合上表现良好,但对未见组合泛化能力差。
- 变压器模型不加结构偏置时无法正确处理否定,凸显系统性推理短板。
神经网络对符号规则的习得与表征能力仍是研究热点。当前工作多聚焦大语言模型在各类推理任务中的惊人表现及其难以理解的失败。本文则转向三种关键神经架构(Transformer、图卷积网络、LSTM)在基于命题逻辑的受控任务中的泛化行为。该任务要求模型为逻辑公式生成满足赋值,提供结构化且可解释的组合性研究场景。我们引入一个平衡扩展的数据集,消除表面模式,支持对未见运算组合的测试。在该数据集上评估三类架构在训练分布外的泛化能力。尽管所有模型在分布内表现良好,但对未见模式,尤其是涉及否定的组合,泛化仍面临显著挑战。变压器模型若无结构偏置,无法进行否定的组合式应用。结果揭示标准架构在学习逻辑算子系统性表征方面存在持续局限,提示需更强归纳偏置以支持鲁棒的基于规则推理。
原文摘要 · Abstract (English)
The extent to which neural networks are able to acquire and represent symbolic rules remains a key topic of research and debate. Much current work focuses on the impressive capabilities of large language models, as well as their often ill-understood failures on a wide range of reasoning tasks. In this paper, in contrast, we investigate the generalization behavior of three key neural architectures (Transformers, Graph Convolution Networks and LSTMs) in a controlled task rooted in propositional logic. The task requires models to generate satisfying assignments for logical formulas, making it a structured and interpretable setting for studying compositionality. We introduce a balanced extension of an existing dataset to eliminate superficial patterns and enable testing on unseen operator combinations. Using this dataset, we evaluate the ability of the three architectures to generalize beyond the training distribution. While all models perform well in-distribution, we find that generalization to unseen patterns, particularly those involving negation, remains a significant challenge. Transformers fail to apply negation compositionally, unless structural biases are introduced. Our findings highlight persistent limitations in the ability of standard architectures to learn systematic representations of logical operators, suggesting the need for stronger inductive biases to support robust rule-based reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。