通过构建对称性约束模型,小数据预训练后可免费扩展至大规模逻辑规则推理。
Pretrain on Small Synthetic Data, Scale Large for Free: Symmetry-Aware Foundation Model for Logic Rule Induction

- 用架构与推理设计强制保持原子命名、样本顺序等对称性,实现无重训练扩展。
- 在合成测试中,大规模场景下支持标签准确率稳定,新输入规则保真度更高。
- 适合需要可解释规则且需跨场景迁移的逻辑推理任务,尤其关注模型泛化能力。
逻辑规则归纳旨在获得可解释且能在命题模式间迁移的规则。这要求尊重对称性:原子命名、样本顺序、极性翻转和标签交换。通过构造方式严格保证对称性,使训练好的归纳器能超越原始训练模式规模。本文核心贡献是提出一种规范输出机制,可从字面分值中解码出离散规则。该机制无需重训练,只要分值满足对称性,其输出即为精确等变。我们在神经规则归纳器(NRI)上实现此方法,该模型原生仅尊重样本顺序。通过架构、推理与训练恢复其余对称性。在合成压力测试中,大模式下的支持标签准确率保持稳定,新输入规则保真度优于未修改模型。真实数据上,大模式下准确率提升显著。导出规则在合成全群测试和有效模式的真实数据测试中均为精确。此性质为导出机制的数学特性,我们仅在NRI上进行实证验证。通过构造性对称性约束,将小数据预训练模型转化为可复用、可解释的归纳器,实现向更大模式的迁移。
原文摘要 · Abstract (English)
Logical rule induction seeks interpretable rules that transfer across propositional schemas. This requires respecting symmetries: atom naming, example order, polarity flips, and label swap. Enforcing exact symmetry by construction lets one trained inducer scale beyond its training schemas. Our central contribution is a canonical export that decodes a discrete rule from literal scores. It needs no retraining and is exactly equivariant whenever those scores respect the symmetries. We instantiate it on the Neural Rule Inducer, a disjunctive-normal-form (DNF) foundation model that natively respects only example order. We restore the remaining symmetries through architecture, inference, and training. On synthetic stress tests, accuracy on the support labels stays stable at much larger schemas, and rule fidelity on fresh inputs remains above the unmodified model. On real data, accuracy improves most on larger schemas. The exported rule is exact on synthetic full-group tests and on schema-valid real-data tests. This is a mathematical property of the export rather than of a specific model, and we validate it empirically only on the NRI. Enforcing symmetry by construction turns this small-data pretrained model into a reusable, interpretable inducer that transfers to larger schemas.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。