用四元数与块循环结构压缩模型,边端部署更高效。
EdgeLDR: Quaternion Low-Displacement Rank Neural Networks for Edge-Efficient Deep Learning
- 四元数通道混合+块循环结构,支持快速FFT计算
- 相比传统方法,大块尺寸下延迟稳定且加速显著
- 适用于轻量CNN与Transformer,适合资源受限设备
将深度神经网络部署在边缘设备常受密集线性算子带来的内存流量和计算开销限制。虽然四元数神经网络通过哈密顿乘积耦合多通道提升了参数效率,但通常仍保留非结构化稠密权重;而结构化矩阵虽可实现快速计算,却多限于实数域。本文提出EdgeLDR框架,实现四元数块循环线性与卷积层,结合四元数通道混合与块循环参数结构,并通过复共轭表示支持基于FFT的求解。我们提供了EdgeLDR层的参考实现,对比了基于FFT的计算与朴素空间域四元数循环乘积实现。结果表明,FFT评估在大块尺寸下保持低延迟并带来显著实际加速,使更大压缩比成为可能。进一步将EdgeLDR层集成至紧凑型CNN与Transformer骨干网络,在32×32彩色图像分类(CIFAR-10/100、SVHN)及高光谱图像分类(Houston 2013、Pavia University)上评估精度-压缩权衡,报告参数量与CPU/GPU延迟。结果表明,EdgeLDR层在保持竞争力准确率的同时实现显著压缩。
原文摘要 · Abstract (English)
Deploying deep neural networks on edge devices is often limited by the memory traffic and compute cost of dense linear operators. While quaternion neural networks improve parameter efficiency by coupling multiple channels through Hamilton products, they typically retain unstructured dense weights; conversely, structured matrices enable fast computation but are usually applied in the real domain. This paper introduces EdgeLDR, a practical framework for quaternion block-circulant linear and convolutional layers that combines quaternion channel mixing with block-circulant parameter structure and enables FFT-based evaluation through the complex adjoint representation. We present reference implementations of EdgeLDR layers and compare FFT-based computation against a naive spatial-domain realization of quaternion circulant products. FFT evaluation yields large empirical speedups over the naive implementation and keeps latency stable as block size increases, making larger compression factors computationally viable. We further integrate EdgeLDR layers into compact CNN and Transformer backbones and evaluate accuracy-compression trade-offs on 32x32 RGB classification (CIFAR-10/100, SVHN) and hyperspectral image classification (Houston 2013, Pavia University), reporting parameter counts and CPU/GPU latency. The results show that EdgeLDR layers provide significant compression with competitive accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。