提出窗口化矩阵乘法，让大卷积核计算更快更高效。
WBMM: Windowed Batch Matrix Multiplication for Efficient Large Receptive Field Convolution
- 将输入分块为连续窗口，用位置偏置表构建权重矩阵，实现规则内存访问
- 14x14窗口比5x5深度卷积快，每层感受野扩大7.8倍
- 适用于多种设备，训练速度提升1.31-1.88倍，无需专用加速内核
大核深度卷积性能强但随核大小增加导致显著退化，源于基于gather的不规则内存访问；而现有LKA在小特征图上有效，大特征图上反而更慢。本文提出窗口化批量矩阵乘法（WBMM），将输入划分为连续窗口，通过紧凑的位置偏置表构建权重矩阵，借助批量矩阵乘法实现规则内存访问。该方法具有独特性质：吞吐量随窗口增大而提升，与深度卷积相反。算子级基准测试显示，使用14x14窗口的WBMM在速度上超越5x5深度卷积基线，且每层感受野扩大7.8倍。结合块间跨窗口通信与分层窗口重参数化，WBMM在ImageNet-1K、COCO和ADE20K上达到相当或更高精度，训练速度提升1.31–1.88倍，并在GPU、CPU及边缘设备上保持一致优势，无需特殊加速内核。代码已开源。
原文摘要 · Abstract (English)
Large kernel depthwise convolutions achieve strong performance but suffer from significant degradation as kernel size grows due to irregular memory access from gather-based computation; while Large Kernel Acceleration (LKA) helps on small feature maps, it becomes counterproductive on large feature maps, even slower than non-accelerated implementations. We propose Windowed Batch Matrix Multiplication (WBMM), which partitions input into contiguous windows and indexes a compact relative position bias table to construct weight matrices, enabling regular memory access via batched matrix multiplication. This yields a unique property: WBMM's throughput improves with larger windows, opposite to depthwise convolutions that degrade with larger kernels. Operator-level benchmarks show WBMM with 14x14 windows outperforms 5x5 depthwise convolution baselines in speed while providing a 7.8x larger per-layer receptive field. Combined with inter-block cross-window communication and hierarchical window reparameterization, WBMM achieves comparable or higher accuracy on ImageNet-1K, COCO, and ADE20K with 1.31-1.88x training speedup, and demonstrates consistent advantages across GPU, CPU, and edge devices without requiring specialized acceleration kernels. Our code is available at https://github.com/wansong-s/WBMM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。