arXiv:2410.13663cs.CV2024-10被引 23

用混合模型提升无人机灾情识别,速度快且准确。

DiRecNetV2: A Transformer-Enhanced Network for Aerial Disaster Recognition

  • 融合卷积与Transformer,兼顾局部特征和全局上下文。
  • 在单标签测试中达0.964加权F1,多标签测试仍保持0.614。
  • 每秒处理176帧,适合无人机实时灾情监测。

将无人机(UAV)与人工智能模型结合用于灾情评估中的航拍图像处理,需要具备高精度、计算高效和实时处理能力的模型。传统卷积神经网络(CNN)在局部特征提取上效率高,但对全局上下文理解能力有限;而视觉变换器(ViT)通过注意力机制可增强全局理解,但在基于无人机的灾情响应应用中仍研究不足。为此,我们提出DiRecNetV2,一种改进的混合模型,融合卷积层与变换器层,结合CNN的归纳偏置以实现鲁棒特征提取,同时利用Transformer的全局上下文理解能力,并保持低计算负载,适用于无人机场景。此外,我们构建了一个新的紧凑型多标签灾情数据集,为未来研究设立基准,探索在单标签数据上训练的模型在多标签测试集上的表现。研究在AIDERSv2数据集上评估轻量级CNN与ViT,以每秒帧数(FPS)衡量效率,以加权F1分数衡量分类性能。DiRecNetV2在单标签测试集上达到0.964的加权F1分数,同时在复杂多标签测试集中表现良好,得分为0.614,且在Nvidia Orin Jetson设备上运行速度达176.13 FPS。

原文摘要 · Abstract (English)

The integration of Unmanned Aerial Vehicles (UAVs) with artificial intelligence (AI) models for aerial imagery processing in disaster assessment, necessitates models that demonstrate exceptional accuracy, computational efficiency, and real-time processing capabilities. Traditionally Convolutional Neural Networks (CNNs), demonstrate efficiency in local feature extraction but are limited by their potential for global context interpretation. On the other hand, Vision Transformers (ViTs) show promise for improved global context interpretation through the use of attention mechanisms, although they still remain underinvestigated in UAV-based disaster response applications. Bridging this research gap, we introduce DiRecNetV2, an improved hybrid model that utilizes convolutional and transformer layers. It merges the inductive biases of CNNs for robust feature extraction with the global context understanding of Transformers, maintaining a low computational load ideal for UAV applications. Additionally, we introduce a new, compact multi-label dataset of disasters, to set an initial benchmark for future research, exploring how models trained on single-label data perform in a multi-label test set. The study assesses lightweight CNNs and ViTs on the AIDERSv2 dataset, based on the frames per second (FPS) for efficiency and the weighted F1 scores for classification performance. DiRecNetV2 not only achieves a weighted F1 score of 0.964 on a single-label test set but also demonstrates adaptability, with a score of 0.614 on a complex multi-label test set, while functioning at 176.13 FPS on the Nvidia Orin Jetson device.

无人机灾情识别混合模型实时处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。