用多项式空间优化逻辑门网络训练，提升深层网络稳定性。
Fitting Multilinear Polynomials for Logic Gate Networks
- 将逻辑门映射到4维多项式空间，降低参数量至每单元4个。
- 新方法在7个数据集上均不逊于基线，深度12层时误差仅-0.5pp。
- 适合追求轻量级、高稳定性的可学习电路设计研究者。
我们研究可学习的逻辑门网络，通过堆叠二输入布尔门构建组合电路。每个二输入门对应一个包含4个系数的多线性多项式，16种布尔门构成4维空间中的原型码本，使训练转化为向量量化问题。基线方法Soft-Mix在门类型上使用16维softmax，但码本秩约为4：15个单纯形方向中有11个携带零梯度，均匀初始化时反向信号完全消失。我们证明，任何仿射乘积重参数化无法解决STE下的交互系数饥饿问题；而软向量量化选择的协方差雅可比（CovJac）通过将饥饿系数与始终活跃的常数通道耦合，绕过该问题。在4维多项式空间中，每个神经元参数从16个减少到4个。在七个数据集上，至少一种4参数方法在所有数据集上表现不低于或超过Soft-Mix；CovJac相比STE的优势随交互需求单调增长。深度方面，Soft-Mix在12层时性能崩溃（CIFAR-10上下降37.3pp），而CovJac保持稳定（CIFAR-10上仅-0.5pp，MNIST上稳定）。
原文摘要 · Abstract (English)
We study learnable logic gate networks that stack layers of 2-input Boolean gates to build combinational circuits. Every 2-input gate has a unique multilinear polynomial with 4 coefficients, so the 16 Boolean gates form a codebook of prototypes in a 4-dimensional space, reducing training to a vector-quantization problem. The baseline method, Soft-Mix, learns a 16-dimensional softmax over gate identities, but the codebook has rank~4: 11 of 15 simplex directions carry nullspace gradient, and at uniform initialization the backward signal vanishes exactly. We prove that no affine product reparameterization fixes the resulting interaction-coefficient starvation under STE, and show that the covariance Jacobian of soft-VQ selection bypasses it by coupling the starved coefficient to the always-active constant channel. Working in the 4-dimensional polynomial space reduces each neuron from 16 to 4 parameters. On seven datasets, at least one 4-parameter method matches or exceeds Soft-Mix on every dataset; the CovJac advantage over STE grows monotonically with interaction demand across all seven datasets. At depth, Soft-Mix collapses ($-37.3$pp on CIFAR-10 at 12 layers) while CovJac holds ($-0.5$pp on CIFAR-10, stable on MNIST).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。