针对低分辨率图像设计新型级联多尺度注意力机制,提升关键点检测精度。
Cascaded Multi-Scale Attention for Enhanced Multi-Scale Feature Extraction and Interaction with Low-Resolution Images
- 提出级联多尺度注意力,无需下采样即可融合多尺度特征。
- 在人体姿态估计等任务上超越现有方法,参数更少且性能更强。
- 适合摄像头捕获低分辨率图像的现实场景,如安防与移动端应用。
在真实图像识别任务中,如人体姿态估计,摄像头常捕捉低分辨率目标(如人体)。这给多尺度特征提取与利用带来挑战,而多尺度特征对精确推理至关重要。为此,我们提出一种新型注意力机制——级联多尺度注意力(CMSA),专为CNN-ViT混合架构设计,以高效处理低分辨率输入。CMSA通过分组多头自注意力与基于窗口的局部注意力相结合,并实现跨尺度特征的级联融合,无需下采样即可提取并无缝整合不同尺度特征。该结构显著增强模型在低分辨率图像下的表现,尤其在人体姿态估计、头部姿态估计等任务中效果突出。实验表明,该方法在性能上优于现有最先进方法,且参数量更少,具备广泛应用于无法获取高分辨率图像的实际场景的潜力。代码已开源:https://github.com/xyongLu/CMSA。
原文摘要 · Abstract (English)
In real-world applications of image recognition tasks, such as human pose estimation, cameras often capture objects, like human bodies, at low resolutions. This scenario poses a challenge in extracting and leveraging multi-scale features, which is often essential for precise inference. To address this challenge, we propose a new attention mechanism, named cascaded multi-scale attention (CMSA), tailored for use in CNN-ViT hybrid architectures, to handle low-resolution inputs effectively. The design of CMSA enables the extraction and seamless integration of features across various scales without necessitating the downsampling of the input image or feature maps. This is achieved through a novel combination of grouped multi-head self-attention mechanisms with window-based local attention and cascaded fusion of multi-scale features over different scales. This architecture allows for the effective handling of features across different scales, enhancing the model's ability to perform tasks such as human pose estimation, head pose estimation, and more with low-resolution images. Our experimental results show that the proposed method outperforms existing state-of-the-art methods in these areas with fewer parameters, showcasing its potential for broad application in real-world scenarios where capturing high-resolution images is not feasible. Code is available at https://github.com/xyongLu/CMSA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。