探索可微逻辑门网络在大规模图像分类中的极限与优化方法
From MNIST to ImageNet: Understanding the Scalability Boundaries of Differentiable Logic Gate Networks
- 用可微逻辑门构建快速低功耗网络，通过硬件友好方式加速推理
- 在2000类图像数据上验证性能，发现温度调节对输出层影响显著
- 提出分组求和输出策略，提升模型在超大规模分类任务中的表现
可微逻辑门网络（DLGNs）是一种快速且节能的前馈网络替代方案。通过可学习的逻辑门组合，实现硬件友好的高效推理。由于该架构近期才受到关注，其设计与扩展性仍处于初期阶段，尤其在输出层方面。此前研究主要限于十类以下的数据集。本文系统考察了DLGN在大型多分类数据集上的表现，分析其表达能力与可扩展性，并评估多种输出策略。基于合成与真实数据集，揭示了温度调节的重要性及其对输出层性能的影响。研究还明确了分组求和层在何种条件下表现优异，并成功将其应用于高达2000类的大规模分类任务。
原文摘要 · Abstract (English)
Differentiable Logic Gate Networks (DLGNs) are a very fast and energy-efficient alternative to conventional feed-forward networks. With learnable combinations of logical gates, DLGNs enable fast inference by hardware-friendly execution. Since the concept of DLGNs has only recently gained attention, these networks are still in their developmental infancy, including the design and scalability of their output layer. To date, this architecture has primarily been tested on datasets with up to ten classes. This work examines the behavior of DLGNs on large multi-class datasets. We investigate its general expressiveness, its scalability, and evaluate alternative output strategies. Using both synthetic and real-world datasets, we provide key insights into the importance of temperature tuning and its impact on output layer performance. We evaluate conditions under which the Group-Sum layer performs well and how it can be applied to large-scale classification of up to 2000 classes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。