提出轻量级动态上采样方法，参数和计算量大幅减少但性能更优。
Lighten CARAFE: Dynamic Lightweight Upsampling with Guided Reassemble Kernels
- 用小规模核空间加可学习偏移动态生成大核，避免大量参数
- 16倍上采样时参数少91%、浮点运算少63%，检测mAP高0.3%
- 适合对模型效率敏感的视觉任务，如目标检测、分割
特征上采样是现代视觉模型中的基础操作，理想方法应轻量且低复杂度，既能提升性能又不增加模型负担。现有内容感知特征重组装（CARAFE）虽表现良好，但需生成大规模可训练核，带来大量冗余参数，扩展性受限。本文提出轻量动态上采样（DLU），先构建小规模源核空间，再通过可学习引导偏移从中采样大核，避免引入大量可训练参数。在多个主流视觉任务上的实验表明，DLU性能与原始CARAFE相当甚至更优，而复杂度显著降低：16倍上采样时参数减少91%，浮点运算减少至少63%，且目标检测mAP高出0.3%。代码已开源。
原文摘要 · Abstract (English)
As a fundamental operation in modern machine vision models, feature upsampling has been widely used and investigated in the literatures. An ideal upsampling operation should be lightweight, with low computational complexity. That is, it can not only improve the overall performance but also not affect the model complexity. Content-aware Reassembly of Features (CARAFE) is a well-designed learnable operation to achieve feature upsampling. Albeit encouraging performance achieved, this method requires generating large-scale kernels, which brings a mass of extra redundant parameters, and inherently has limited scalability. To this end, we propose a lightweight upsampling operation, termed Dynamic Lightweight Upsampling (DLU) in this paper. In particular, it first constructs a small-scale source kernel space, and then samples the large-scale kernels from the kernel space by introducing learnable guidance offsets, hence avoiding introducing a large collection of trainable parameters in upsampling. Experiments on several mainstream vision tasks show that our DLU achieves comparable and even better performance to the original CARAFE, but with much lower complexity, e.g., DLU requires 91% fewer parameters and at least 63% fewer FLOPs (Floating Point Operations) than CARAFE in the case of 16x upsampling, but outperforms the CARAFE by 0.3% mAP in object detection. Code is available at https://github.com/Fu0511/Dynamic-Lightweight-Upsampling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。