在极低内存下实现高效高维分类,精度损失极小。
DecoHD: Decomposed Hyperdimensional Classification under Extreme Memory Budgets
- 将高维计算分解为轻量共享通道,跨层乘法绑定+末端捆绑
- 参数量减少97%仍保持接近基准的精度(最差仅降5.7%)
- 适合部署在内存受限的加速器,能效提升超200倍
分解是压缩深度网络而不改变输入输出维度或接口语义的有效方法。本文将其引入高维计算(HDC),传统方法通过固定原子高维向量解码,不适用于压缩学习到的类别原型。我们提出DecoHD,直接在分解的HDC参数化下进行学习:每层使用少量共享通道,跨层采用乘法绑定,末尾进行捆绑,从紧凑因子生成大表示空间。通过轻量捆绑头压缩类别轴,在保留原生绑定-捆绑-评分机制的同时实现端到端训练,推理仍为纯HDC,契合近/内存加速器。实验表明,DecoHD在极端内存预算下实现显著压缩,平均仅比强基线低0.1–0.15%(最差5.7%),更抗随机位翻转噪声,以约97%更少可训练参数达到精度峰值;硬件测试中相较CPU(AMD Ryzen 9 9950X)实现约277倍/35倍能效/速度提升,相较GPU(NVIDIA RTX 4090)达13.5倍/3.7倍,相较基线HDC ASIC达2.0倍/2.4倍。
原文摘要 · Abstract (English)
Decomposition is a proven way to shrink deep networks without changing input-output dimensionality or interface semantics. We bring this idea to hyperdimensional computing (HDC), where footprint cuts usually shrink the feature axis and erode concentration and robustness. Prior HDC decompositions decode via fixed atomic hypervectors, which are ill-suited for compressing learned class prototypes. We introduce DecoHD, which learns directly in a decomposed HDC parameterization: a small, shared set of per-layer channels with multiplicative binding across layers and bundling at the end, yielding a large representational space from compact factors. DecoHD compresses along the class axis via a lightweight bundling head while preserving native bind-bundle-score; training is end-to-end, and inference remains pure HDC, aligning with in/near-memory accelerators. In evaluation, DecoHD attains extreme memory savings with only minor accuracy degradation under tight deployment budgets. On average it stays within about 0.1-0.15% of a strong non-reduced HDC baseline (worst case 5.7%), is more robust to random bit-flip noise, reaches its accuracy plateau with up to ~97% fewer trainable parameters, and--in hardware--delivers roughly 277x/35x energy/speed gains over a CPU (AMD Ryzen 9 9950X), 13.5x/3.7x over a GPU (NVIDIA RTX 4090), and 2.0x/2.4x over a baseline HDC ASIC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。