用沃尔什变换加速逻辑门学习，实现低资源高效率的神经网络训练。
WARP-LUTs -- Walsh-Assisted Relaxation for Probabilistic Look Up Tables
- 引入沃尔什变换优化概率查找表的梯度学习过程。
- 在CIFAR-10上收敛速度远超DLGN，参数量更少且精度相当。
- 适合FPGA部署与实时科学计算，适用于低功耗硬件场景。
高效机器学习日益受到关注，推动了新型模型架构与软硬件协同设计的发展。近期的软硬件协同方法已实现完全无乘法运算的模型，表现优异。例如，可微逻辑门网络（DLGNs）通过梯度优化学习底层逻辑门的最佳组合，在准确率、资源占用和延迟之间取得了当前最优平衡。然而，这些模型在训练时计算成本高，且难以泛化到多输入逻辑块。本文提出沃尔什辅助松弛的概率查找表（WARP-LUTs），一种新的基于梯度的方法，能以极少的可训练参数高效学习逻辑门组合。实验表明，WARP-LUTs在CIFAR-10上的收敛速度显著优于DLGNs，同时保持相近的精度。该方法还展现出向更高输入逻辑块扩展的潜力，为现代FPGA上的极低资源部署及实时科学应用提供了新方向。
原文摘要 · Abstract (English)
Fast and efficient machine learning is of growing interest to the scientific community and has spurred significant research into novel model architectures and hardware-aware design. Recent hard? and software co-design approaches have demonstrated impressive results with entirely multiplication-free models. Differentiable Logic Gate Networks (DLGNs), for instance, provide a gradient-based framework for learning optimal combinations of low-level logic gates, setting state-of-the-art trade-offs between accuracy, resource usage, and latency. However, these models suffer from high computational cost during training and do not generalize well to logic blocks with more inputs. In this work, we introduce Walsh-Assisted Relaxation for Probabilistic Look-Up Tables (WARP-LUTs) - a novel gradient-based method that efficiently learns combinations of logic gates with substantially fewer trainable parameters. We demonstrate that WARP-LUTs achieve significantly faster convergence on CIFAR-10 compared to DLGNs, while maintaining comparable accuracy. Furthermore, our approach suggests potential for extension to higher-input logic blocks, motivating future research on extremely efficient deployment on modern FPGAs and its real-time science applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。