arXiv:2605.16443cs.LGcs.AI2026-05

用两种权重构造循环对称矩阵,极大压缩模型体积

Two-Valued Symmetric Circulant Matrices: Applications in Deep Learning

论文配图:Two-Valued Symmetric Circulant Matrices: Applications in Deep Learning
图 1 · 摘自论文原文
  • 每层仅用两个共享权重构建循环对称结构
  • 参数量减少超80倍,MNIST从62万降至7852
  • 适合边缘设备、物联网医疗等低功耗场景

尽管深度神经网络在视觉、医学诊断和物联网场景中取得成功,但其在资源受限平台的部署面临存储需求高、计算复杂度大和占用空间大的挑战。特别是全连接层需要大量权重,使边缘设备难以承载。为此,本文提出两值对称循环矩阵(TVSCM),一种极稀疏架构,每层仅使用两个权重维持循环与对称性。该结构大幅降低存储成本,无需硬件改造或传统稀疏学习技术(如低秩近似、剪枝)的额外步骤。仿真表明,在MNIST上参数量从623,290降至7,852,减少超过80倍;在MIT-BIH心律失常数据集上从24,709降至942,同时保持精度:MNIST从97.6%降至93.5%,MIT-BIH从97.6%降至93.1%。因其极低的架构要求和功耗,适用于边缘计算、微型机器学习、物联网医疗及电池供电系统。

原文摘要 · Abstract (English)

Despite the success of deep neural networks in vision, medical diagnosis, and IoT scenarios, their deployment on resource-limited platforms poses serious challenges due to their high storage requirements, computational complexity, and large footprint. In particular, fully connected layers require a large number of weights, making it difficult for edge devices to accommodate them. To overcome these challenges associated with limited platforms, this paper proposes the Two-Valued Symmetric Circulant Matrix (TVSCM), a very sparse architecture that employs just two weights per layer to keep it circulant and symmetric. The extreme form of structured sparse architecture provides negligible storage costs compared to traditional full-weight storage. Instead of hardware and additional stages of other traditional sparse learning techniques, such as low-rank approximation and pruning approaches, this architecture provides an extreme form of sparsity, achieving very minimal storage requirements. The simulation study demonstrates more than 80$\times$ reduction in model parameters, reducing parameters from 623,290 to 7,852 on MNIST and from 24,709 to 942 on the MIT-BIH arrhythmia dataset, while maintaining comparable accuracy from 97.6% to 93.5% on MNIST and from 97.6% to 93.1% on MIT-BIH. Due to its minimal architectural requirements and very low power consumption, this architecture would be ideal for edge computing platforms, tiny-ML platforms, IoMT systems, and battery-powered systems.

稀疏架构边缘计算模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。