降低向量维度100倍的同时保持精度，提升低维符号架构训练稳定性。
Towards Vector Optimization on Low-Dimensional Vector Symbolic Architecture
- 采用基于梯度的优化方法，在低维下实现稳定自适应更新。
- 批量归一化与知识蒸馏使准确率提升，且推理无额外开销。
- 适用于追求高效低维神经网络的模型部署与可解释性研究。
向量符号架构（VSA）因高效性在机器学习中崭露头角，但受限于高维性和准确性问题。低维计算（LDC）方法通过梯度优化将向量维度降低约100倍，同时保持精度，具有巨大潜力。然而，其在VSA中的优化仍待深入探索。我们发现，稳定且自适应的向量更新动态对LDC训练至关重要。此外，批量归一化（BN）和知识蒸馏（KD）在常规方法中被忽视却极为关键：BN不增加推理计算开销，而KD显著提升推理置信度。在多个基准测试上的广泛实验与消融分析表明，该方法不仅提升了性能，还扩展了类似LDC的二值神经网络优化的可解释性，填补了现有二值网络文献的空白。
原文摘要 · Abstract (English)
Vector Symbolic Architecture (VSA) is emerging in machine learning due to its efficiency, but they are hindered by issues of hyperdimensionality and accuracy. As a promising mitigation, the Low-Dimensional Computing (LDC) method significantly reduces the vector dimension by ~100 times while maintaining accuracy, by employing a gradient-based optimization. Despite its potential, LDC optimization for VSA is still underexplored. Our investigation into vector updates underscores the importance of stable, adaptive dynamics in LDC training. We also reveal the overlooked yet critical roles of batch normalization (BN) and knowledge distillation (KD) in standard approaches. Besides the accuracy boost, BN does not add computational overhead during inference, and KD significantly enhances inference confidence. Through extensive experiments and ablation studies across multiple benchmarks, we provide a thorough evaluation of our approach and extend the interpretability of binary neural network optimization similar to LDC, previously unaddressed in BNN literature.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。