通过解耦适配器实现动态卷积核共享,提升模型效率与精度。
KernelDNA: Dynamic Kernel Sharing via Decoupled Naive Adapters
- 用适配器解耦输入依赖路由与预训练调制,实现轻量级动态核生成
- 在图像分类与密集预测任务中达到动态卷积领先精度-效率平衡
- 适合追求高效部署的视觉模型优化场景
动态卷积通过自适应组合多个卷积核提升模型容量,但面临显著权衡:现有方法或因线性扩展核数导致参数开销大,或因复杂核交互降低推理速度,或难以联合优化动态注意力与静态卷积核。我们发现预训练卷积神经网络存在层间冗余,类似大语言模型中的冗余现象。具体而言,密集卷积层可被由共享父核经适配器生成的“子”层高效替代。为此,我们提出轻量级卷积核插件KernelDNA,将核适配解耦为输入依赖的动态路由与预训练静态调制,兼顾参数效率与硬件友好推理。不同于通过多核集成扩大参数量的现有方法,本方法采用跨层权重共享与适配器调制,实现动态核专化而不改变标准卷积结构。该设计在保持标准卷积原生计算效率的同时,通过输入自适应核调整增强表征能力。图像分类与密集预测任务的实验表明,KernelDNA在动态卷积变体中实现了最先进的精度-效率平衡。
原文摘要 · Abstract (English)
Dynamic convolution enhances model capacity by adaptively combining multiple kernels, yet faces critical trade-offs: prior works either (1) incur significant parameter overhead by scaling kernel numbers linearly, (2) compromise inference speed through complex kernel interactions, or (3) struggle to jointly optimize dynamic attention and static kernels. We observe that pre-trained Convolutional Neural Networks (CNNs) exhibit inter-layer redundancy akin to that in Large Language Models (LLMs). Specifically, dense convolutional layers can be efficiently replaced by derived "child" layers generated from a shared "parent" convolutional kernel through an adapter. To address these limitations and implement the weight-sharing mechanism, we propose a lightweight convolution kernel plug-in, named KernelDNA. It decouples kernel adaptation into input-dependent dynamic routing and pre-trained static modulation, ensuring both parameter efficiency and hardware-friendly inference. Unlike existing dynamic convolutions that expand parameters via multi-kernel ensembles, our method leverages cross-layer weight sharing and adapter-based modulation, enabling dynamic kernel specialization without altering the standard convolution structure. This design preserves the native computational efficiency of standard convolutions while enhancing representation power through input-adaptive kernel adjustments. Experiments on image classification and dense prediction tasks demonstrate that KernelDNA achieves a state-of-the-art accuracy-efficiency balance among dynamic convolution variants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。