arXiv:2409.19648cs.CV2024-09中稿 · IEEE Transactions …被引 79

用Transformer做遥感图像中任意朝向目标检测,精度提升且训练更快。

OrientedFormer: An End-to-End Transformer-Based Oriented Object Detector in Remote Sensing Images

  • 用高斯分布编码目标角度、位置和大小,解决旋转信息缺失问题。
  • 在DOTA等6个数据集上,相比之前方法提升1.21的AP50,训练只需1倍周期。
  • 适合需要端到端检测、关注几何关系的遥感图像分析任务。

遥感图像中的定向目标检测因目标分布具有多方向性而具有挑战性。近期基于Transformer的端到端方法通过消除后处理步骤,在性能上优于传统基于CNN的方法。然而,直接将Transformer扩展到定向目标检测面临三大问题:1)目标任意旋转,需同时编码角度、位置与尺寸;2)自注意力机制缺乏几何关系,因内容查询与位置查询间缺乏交互;3)定向目标导致错位,主要出现在交叉注意力中值与位置查询之间的对齐困难,影响分类与定位精度。本文提出一种端到端的Transformer定向目标检测器,包含三个专用模块:首先,提出高斯位置编码,利用高斯分布编码定向框的角度、位置与尺寸;其次,提出Wasserstein自注意力,通过高斯Wasserstein距离得分引入几何关系,促进内容与位置查询间的交互;第三,提出定向交叉注意力,通过按角度旋转采样点来对齐值与位置查询。在六个数据集DIOR-R、一系列DOTA、HRSC2016和ICDAR2015上的实验表明本方法有效。相较于先前端到端检测器,OrientedFormer在DIOR-R和DOTA-v1.0上分别获得1.16和1.21的AP50提升,训练周期从3×减少至1×。代码已公开于https://github.com/wokaikaixinxin/OrientedFormer。

原文摘要 · Abstract (English)

Oriented object detection in remote sensing images is a challenging task due to objects being distributed in multi-orientation. Recently, end-to-end transformer-based methods have achieved success by eliminating the need for post-processing operators compared to traditional CNN-based methods. However, directly extending transformers to oriented object detection presents three main issues: 1) objects rotate arbitrarily, necessitating the encoding of angles along with position and size; 2) the geometric relations of oriented objects are lacking in self-attention, due to the absence of interaction between content and positional queries; and 3) oriented objects cause misalignment, mainly between values and positional queries in cross-attention, making accurate classification and localization difficult. In this paper, we propose an end-to-end transformer-based oriented object detector, consisting of three dedicated modules to address these issues. First, Gaussian positional encoding is proposed to encode the angle, position, and size of oriented boxes using Gaussian distributions. Second, Wasserstein self-attention is proposed to introduce geometric relations and facilitate interaction between content and positional queries by utilizing Gaussian Wasserstein distance scores. Third, oriented cross-attention is proposed to align values and positional queries by rotating sampling points around the positional query according to their angles. Experiments on six datasets DIOR-R, a series of DOTA, HRSC2016 and ICDAR2015 show the effectiveness of our approach. Compared with previous end-to-end detectors, the OrientedFormer gains 1.16 and 1.21 AP$_{50}$ on DIOR-R and DOTA-v1.0 respectively, while reducing training epochs from 3$\times$ to 1$\times$. The codes are available at https://github.com/wokaikaixinxin/OrientedFormer.

遥感检测定向目标Transformer端到端

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。