用自适应交叉哈达玛积提升视觉模型效率与表达力
Expressive yet Efficient Feature Expansion with Adaptive Cross-Hadamard Products
- 引入自适应交叉哈达玛模块，通过可微离散采样实现高效特征扩展
- 在不增加卷积参数前提下达成更高精度与更快推理速度
- 适合资源受限场景下的高效视觉模型设计与部署
近期理论研究揭示了哈达玛积能生成非线性表征并隐式映射至高维空间，但在资源受限的视觉模型中尚缺乏实际应用。为此，我们提出自适应交叉哈达玛（ACH）模块，通过可微离散采样和动态软符号归一化引入可学习性，实现无需额外卷积参数的高效特征复用，同时保证梯度稳定传播。将该模块集成至通过神经架构搜索优化的Hadamptive-Net中，显著提升了模型效率。大量实验表明，在图像分类任务上，该方法实现了前所未有的精度/速度权衡，确立了哈达玛操作作为高效视觉模型的关键构建单元。
原文摘要 · Abstract (English)
Recent theoretical advances reveal that the Hadamard product induces nonlinear representations and implicit high-dimensional mappings for the field of deep learning, yet their practical deployment in resource-constrained vision models remains largely unexplored. To address this gap, we introduce the Adaptive Cross-Hadamard (ACH) module, a novel operator that embeds learnability through differentiable discrete sampling and dynamic softsign normalization. This facilitates highly efficient feature reuse without incurring additional convolutional parameters, while ensuring stable gradient flow. Integrated into Hadaptive-Net (Hadamard Adaptive Network) via neural architecture search, our approach achieves unprecedented efficiency. Comprehensive experiments demonstrate state-of-the-art accuracy/speed trade-offs on image classification tasks, establishing Hadamard operations as specific building blocks for efficient vision models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。