arXiv:2510.12565cs.CV2025-10NeurIPS被引 4

首个无人机多光谱多目标追踪基准,提升小目标与密集场景追踪精度

MMOT: The First Challenging Benchmark for Drone-based Multispectral Multi-Object Tracking

  • 构建多光谱影像追踪新基准,支持小目标与遮挡等复杂场景
  • 在488.8万标注上验证,多光谱输入显著优于传统RGB方法
  • 适合无人机视觉、多模态感知及智能交通研究者参考

基于无人机的多目标追踪因目标小、遮挡严重和背景杂乱而极具挑战。现有基于RGB的追踪算法依赖颜色纹理等空间外观特征,在航拍视角下常失效。多光谱影像通过像素级光谱反射率提供关键判别信息,可增强复杂条件下的目标区分能力。然而,专用多光谱无人机数据集的缺失阻碍了该领域发展。为此,我们提出首个挑战性基准MMOT,具备三大特性:(i) 大规模——125段视频序列,覆盖8类物体,含超过488.8万标注;(ii) 全面挑战——涵盖极小目标、高密度、严重遮挡与复杂运动等多种场景;(iii) 精确定向标注——支持航拍视角下的精准定位与歧义减少。为更好提取光谱特征并利用定向标注,我们进一步设计多光谱定向感知追踪方案,包括:(i) 轻量级光谱3D-Stem,整合光谱信息且兼容RGB预训练;(ii) 定向卡尔曼滤波器,实现精确状态估计;(iii) 端到端定向自适应变压器。在代表性追踪器上的大量实验表明,多光谱输入显著提升追踪性能,尤其对小目标和密集物体效果突出。我们相信本工作将推动无人机多光谱多目标追踪研究。项目代码与数据集已开源于https://github.com/Annzstbl/MMOT。

原文摘要 · Abstract (English)

Drone-based multi-object tracking is essential yet highly challenging due to small targets, severe occlusions, and cluttered backgrounds. Existing RGB-based tracking algorithms heavily depend on spatial appearance cues such as color and texture, which often degrade in aerial views, compromising reliability. Multispectral imagery, capturing pixel-level spectral reflectance, provides crucial cues that enhance object discriminability under degraded spatial conditions. However, the lack of dedicated multispectral UAV datasets has hindered progress in this domain. To bridge this gap, we introduce MMOT, the first challenging benchmark for drone-based multispectral multi-object tracking. It features three key characteristics: (i) Large Scale - 125 video sequences with over 488.8K annotations across eight categories; (ii) Comprehensive Challenges - covering diverse conditions such as extreme small targets, high-density scenarios, severe occlusions, and complex motion; and (iii) Precise Oriented Annotations - enabling accurate localization and reduced ambiguity under aerial perspectives. To better extract spectral features and leverage oriented annotations, we further present a multispectral and orientation-aware MOT scheme adapting existing methods, featuring: (i) a lightweight Spectral 3D-Stem integrating spectral features while preserving compatibility with RGB pretraining; (ii) an orientation-aware Kalman filter for precise state estimation; and (iii) an end-to-end orientation-adaptive transformer. Extensive experiments across representative trackers consistently show that multispectral input markedly improves tracking performance over RGB baselines, particularly for small and densely packed objects. We believe our work will advance drone-based multispectral multi-object tracking research. Our MMOT, code, and benchmarks are publicly available at https://github.com/Annzstbl/MMOT.

多光谱追踪无人机监控多目标跟踪遥感感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。