用约束满足方法训练全±1值的神经网络,实现可解释的逻辑计算。
Learning with Boolean threshold functions
- 用布尔阈值函数和约束分解替代传统损失优化
- 在乘法电路等任务中达成精确解或强泛化能力
- 适合需要可解释性与高效推理的离散神经系统
我们提出一种在布尔数据上训练神经网络的方法,所有节点值严格为±1,模型通常等价于非零权重也为±1的网络。该方法将损失最小化替换为非凸约束形式,每个节点实现布尔阈值函数(BTF),训练通过分解为两个互补约束:一是输入、权重与输出间的局部BTF一致性;二是架构一致性,即神经元输出等于下游输入,并在不同训练实例间强制权重相等。使用反射-反射-松弛(RRR)投影算法协调这些约束。每个BTF约束包含一个边缘下界,当该边界足够大时,学习到的表示在理论上稀疏且等价于由±1权重逻辑门构成的网络。在乘法电路发现、二值自编码、逻辑网络推断及细胞自动机学习等任务中,该方法在标准梯度方法失效的区域仍能获得精确解或强泛化性能。结果表明,基于投影的约束满足为离散神经系统学习提供了可行且概念上不同的基础,对可解释性和高效推理具有启示意义。
原文摘要 · Abstract (English)
We develop a method for training neural networks on Boolean data in which the values at all nodes are strictly $\pm 1$, and the resulting models are typically equivalent to networks whose nonzero weights are also $\pm 1$. The method replaces loss minimization with a nonconvex constraint formulation. Each node implements a Boolean threshold function (BTF), and training is expressed through a divide-and-concur decomposition into two complementary constraints: one enforces local BTF consistency between inputs, weights, and output; the other imposes architectural concurrence, equating neuron outputs with downstream inputs and enforcing weight equality across training-data instantiations of the network. The reflect-reflect-relax (RRR) projection algorithm is used to reconcile these constraints. Each BTF constraint includes a lower bound on the margin. When this bound is sufficiently large, the learned representations are provably sparse and equivalent to networks composed of simple logical gates with $\pm 1$ weights. Across a range of tasks -- including multiplier-circuit discovery, binary autoencoding, logic-network inference, and cellular automata learning -- the method achieves exact solutions or strong generalization in regimes where standard gradient-based methods struggle. These results demonstrate that projection-based constraint satisfaction provides a viable and conceptually distinct foundation for learning in discrete neural systems, with implications for interpretability and efficient inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。