无需权重的神经网络，用查表法实现高效边缘计算。
Differentiable Weightless Neural Networks
- 用查找表替代传统权重，通过可微分技术训练
- 在FPGA上延迟更低、能效更高，内存占用更小
- 适合资源受限的嵌入式设备，如微控制器和超低成本芯片
我们提出可微分无权重神经网络（DWN），基于互连查找表构建。通过一种新的扩展有限差分技术，实现对二值输出的近似求导，从而支持训练。引入可学习映射、可学习归约和谱正则化进一步提升模型精度与效率。我们在三种边缘计算场景下评估：（1）FPGA硬件加速器中，相比现有方案，延迟更低、吞吐量更高、能效更好、模型面积更小；（2）低功耗微控制器中，在严格内存限制下表现优于XGBoost；（3）超低成本芯片中，准确率与硬件面积均优于小型模型。在表格数据集上也优于主流方法，平均排名更高。整体表明DWN是面向边缘高吞吐神经网络的开创性方案。
原文摘要 · Abstract (English)
We introduce the Differentiable Weightless Neural Network (DWN), a model based on interconnected lookup tables. Training of DWNs is enabled by a novel Extended Finite Difference technique for approximate differentiation of binary values. We propose Learnable Mapping, Learnable Reduction, and Spectral Regularization to further improve the accuracy and efficiency of these models. We evaluate DWNs in three edge computing contexts: (1) an FPGA-based hardware accelerator, where they demonstrate superior latency, throughput, energy efficiency, and model area compared to state-of-the-art solutions, (2) a low-power microcontroller, where they achieve preferable accuracy to XGBoost while subject to stringent memory constraints, and (3) ultra-low-cost chips, where they consistently outperform small models in both accuracy and projected hardware area. DWNs also compare favorably against leading approaches for tabular datasets, with higher average rank. Overall, our work positions DWNs as a pioneering solution for edge-compatible high-throughput neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。