用双曲几何优化二值神经网络，提升轻量设备部署性能
Hyperbolic Binary Neural Network
- 将二值化约束问题映射到双曲空间，转为无约束优化
- 在CIFAR10/100和ImageNet上精度优于现有方法
- 适合移动端低精度模型部署场景
二值神经网络（BNN）将全精度权重和激活转换为1比特表示，特别适合在轻量级移动设备上部署。传统BNN通常作为约束优化问题在二值空间中求解，而普通神经网络则在连续空间中进行无约束优化。本文提出双曲二值神经网络（HBNN），利用双曲几何框架优化该约束问题。具体地，通过黎曼指数映射将双曲空间中的约束问题转化为欧氏空间中的无约束问题。同时提出指数参数聚类（EPC）方法，相比黎曼指数映射，基于微分同胚缩小了参数域，提高了权重翻转概率，从而最大化BNN的信息增益。在CIFAR10、CIFAR100和ImageNet分类数据集上，使用VGGsmall、ResNet18和ResNet34模型的实验表明，所提HBNN显著优于当前最优方法。
原文摘要 · Abstract (English)
Binary Neural Network (BNN) converts full-precision weights and activations into their extreme 1-bit counterparts, making it particularly suitable for deployment on lightweight mobile devices. While binary neural networks are typically formulated as a constrained optimization problem and optimized in the binarized space, general neural networks are formulated as an unconstrained optimization problem and optimized in the continuous space. This paper introduces the Hyperbolic Binary Neural Network (HBNN) by leveraging the framework of hyperbolic geometry to optimize the constrained problem. Specifically, we transform the constrained problem in hyperbolic space into an unconstrained one in Euclidean space using the Riemannian exponential map. On the other hand, we also propose the Exponential Parametrization Cluster (EPC) method, which, compared to the Riemannian exponential map, shrinks the segment domain based on a diffeomorphism. This approach increases the probability of weight flips, thereby maximizing the information gain in BNNs. Experimental results on CIFAR10, CIFAR100, and ImageNet classification datasets with VGGsmall, ResNet18, and ResNet34 models illustrate the superior performance of our HBNN over state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。