提出多项式代理训练法,让神经网络高效学习三值逻辑门。
Polynomial Surrogate Training for Differentiable Ternary Logic Gate Networks
- 用二次多项式替代传统门枚举,参数量减少2187倍
- 三值网络训练速度比二值快2-3倍,且能发现多样函数门
- 未知输出可作为最优不确定性标识,提升低置信预测的准确率
可微逻辑门网络(DLGN)通过梯度训练学习紧凑可解释的布尔电路,但现有方法仅限于16种两输入二值门。扩展至三值克莱恩逻辑(K₃)并训练可微三值逻辑门网络(DTLGN),使神经元在不确定时能合理放弃判断,具有重要意义。然而,每个神经元的潜在门组合数激增至19,683,导致传统基于softmax的门选择方法不可行。本文提出多项式代理训练(PST),将每个三值神经元表示为9个可学习系数的(2,2)次多项式,实现2,187倍的参数压缩,并证明训练网络与离散逻辑电路间的差距由一个数据无关的承诺损失控制,该损失在收敛时消失。在CIFAR-10上从48K到512K神经元的缩放实验表明,过参数化下该硬化差距持续缩小。三值网络训练速度为二值网络的2-3倍,且能发现功能多样的真实三值门。在合成和表格任务中,未知输出表现如贝叶斯最优不确定性代理,支持选择性预测——当剔除低置信预测后,三值电路精度超越二值模型。更广泛地,PST建立了一种通用的多项式代理方法,其参数增长仅随逻辑取值数的平方增加,为多值可微逻辑开辟了道路。
原文摘要 · Abstract (English)
Differentiable logic gate networks (DLGNs) learn compact, interpretable Boolean circuits via gradient-based training, but all existing variants are restricted to the 16 two-input binary gates. Extending DLGNs to Ternary Kleene $K_3$ logic and training DTLGNs where the UNKNOWN state enables principled abstention under uncertainty is desirable. However, the support set of potential gates per neuron explodes to $19{,}683$, making the established softmax-over-gates training approach intractable. We introduce Polynomial Surrogate Training (PST), which represents each ternary neuron as a degree-$(2,2)$ polynomial with 9 learnable coefficients (a $2{,}187\times$ parameter reduction) and prove that the gap between the trained network and its discretized logic circuit is bounded by a data-independent commitment loss that vanishes at convergence. Scaling experiments from 48K to 512K neurons on CIFAR-10 demonstrate that this hardening gap contracts with overparameterization. Ternary networks train $2$-$3\times$ faster than binary DLGNs and discover true ternary gates that are functionally diverse. On synthetic and tabular tasks we find that the UNKNOWN output acts as a Bayes-optimal uncertainty proxy, enabling selective prediction in which ternary circuits surpass binary accuracy once low-confidence predictions are filtered. More broadly, PST establishes a general polynomial-surrogate methodology whose parameterization cost grows only quadratically with logic valence, opening the door to many-valued differentiable logic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。