提出神经网络实现组合泛化的充要条件，揭示模型结构与训练数据的关键作用。
A Theoretical Analysis of Compositional Generalization in Neural Networks: A Necessary and Sufficient Condition
- 要求计算图匹配真实组合结构，组件编码信息量恰到好处。
- 理论证明该条件对组合泛化既必要又充分，可指导模型设计。
- 适用于评估模型组合泛化能力，尤其适合架构设计与训练策略优化者。
组合泛化是人工智能中的关键属性，使模型能处理已知组件的新组合。尽管多数深度学习模型缺乏此能力，但某些模型在特定任务中表现良好，暗示存在决定性条件。本文推导出神经网络实现组合泛化的必要且充分条件：(i) 计算图需匹配真实的组合结构，(ii) 组件在训练中需编码适量信息。该条件经数学证明，并融合了架构设计、正则化与训练数据特性。通过一个精心设计的最小示例，直观阐释了该条件。此外，讨论了该准则在训练前评估组合泛化潜力的可能性。本研究为神经网络组合泛化提供了基础性理论分析。
原文摘要 · Abstract (English)
Compositional generalization is a crucial property in artificial intelligence, enabling models to handle novel combinations of known components. While most deep learning models lack this capability, certain models succeed in specific tasks, suggesting the existence of governing conditions. This paper derives a necessary and sufficient condition for compositional generalization in neural networks. Conceptually, it requires that (i) the computational graph matches the true compositional structure, and (ii) components encode just enough information in training. The condition is supported by mathematical proofs. This criterion combines aspects of architecture design, regularization, and training data properties. A carefully designed minimal example illustrates an intuitive understanding of the condition. We also discuss the potential of the condition for assessing compositional generalization before training. This work is a fundamental theoretical study of compositional generalization in neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。