用KAN替换特征融合中的线性层,提升视觉变换器姿态估计性能
KAN-FPN-Stem:A KAN-Enhanced Feature Pyramid Stem for Boosting ViT-based Pose Estimation
- 用KAN替代传统FPN末端的线性卷积,增强多尺度特征融合能力
- 在COCO数据集上相比轻量级ViTPose-S提升2.0 AP
- 揭示了视觉变换器前端瓶颈在于特征融合而非注意力模块
视觉变换器(ViT)在姿态估计等密集预测任务中表现出显著潜力,但其性能常受限于如ViTPose中过于简单的前处理设计。这种基础的分块机制难以有效处理多尺度变化,并在初始特征提取阶段造成不可逆的信息损失。为此,本文提出一种新型的KAN增强型FPN-Stem架构。通过严格的消融实验,我们发现性能提升的真正瓶颈并非插件式注意力模块(如CBAM),而在于FPN中的后融合非线性平滑步骤。基于此洞察,核心创新在于保留经典的“上采样加法”融合路径,但将末端标准的3x3线性平滑卷积替换为强大的KAN卷积层。借助其卓越的非线性建模能力,该KAN层可自适应学习并修正多尺度融合过程中产生的“伪影”。在COCO数据集上的大量实验表明,所提出的KAN-FPN-Stem相较轻量级基线ViTPose-S实现高达+2.0 AP的性能提升。本工作不仅提供了一个即插即用的高性能模块,更重要的是揭示:视觉变换器前端的性能瓶颈往往不在于‘特征精炼’(注意力),而在于‘特征融合’(融合)的质量。同时,它通过引入KAN算子,为解决该瓶颈提供了有效路径。
原文摘要 · Abstract (English)
Vision Transformers (ViT) have demonstrated significant promise in dense prediction tasks such as pose estimation. However, their performance is frequently constrained by the overly simplistic front-end designs employed in models like ViTPose. This naive patchification mechanism struggles to effectively handle multi-scale variations and results in irreversible information loss during the initial feature extraction phase. To overcome this limitation, we introduce a novel KAN-enhanced FPN-Stem architecture. Through rigorous ablation studies, we first identified that the true bottleneck for performance improvement lies not in plug-and-play attention modules (e.g., CBAM), but in the post-fusion non-linear smoothing step within the FPN. Guided by this insight, our core innovation is to retain the classic "upsample-and-add" fusion stream of the FPN, but replace its terminal, standard linear 3x3 smoothing convolution with a powerful KAN-based convolutional layer. Leveraging its superior non-linear modeling capabilities, this KAN-based layer adaptively learns and rectifies the "artifacts" generated during the multi-scale fusion process. Extensive experiments on the COCO dataset demonstrate that our KAN-FPN-Stem achieves a significant performance boost of up to +2.0 AP over the lightweight ViTPose-S baseline. This work not only delivers a plug-and-play, high-performance module but, more importantly, reveals that: the performance bottleneck in ViT front-end often lies not in 'feature refinement' (Attention), but in the quality of 'feature fusion' (Fusion). Furthermore, it provides an effective path to address this bottleneck through the introduction of the KAN operator.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。