用Mamba模型实现低功耗夜间无人机跟踪,效果超前且更省资源。
MambaNUT: Nighttime UAV Tracking via Mamba-based Adaptive Curriculum Learning
- 用状态空间模型替代自注意力,线性复杂度提升效率
- 在多个夜间无人机数据集上达到顶尖性能,参数量更低
- 自适应课程学习动态调参,适合小样本夜间追踪场景
结合低光增强与域适应,夜间无人机跟踪已取得显著进展。然而,过度依赖图像增强、高质量夜间数据有限,以及日间与夜间跟踪器的脱节,制约了端到端可训练框架的发展。此外,现有基于ViT的跟踪器因依赖自注意力机制而计算开销巨大。本文提出一种纯Mamba-based跟踪框架(MambaNUT),采用具有线性复杂度的状态空间模型作为主干,设计单流结构将特征学习与模板搜索耦合于Vision Mamba中。引入自适应课程学习(ACL)方法,动态调整采样策略与损失权重,增强模型泛化能力。ACL包含两级调度器:(1) 采样调度器,将数据分布从不平衡转为平衡,并由易(白天)到难(夜间)样本渐进;(2) 损失调度器,根据训练集规模和实例交并比(IoU)动态分配权重。在多个夜间无人机跟踪基准上的实验证明,所提MambaNUT在保持更低计算成本的前提下达到最先进性能。代码将公开于https://github.com/wuyou3474/MambaNUT。
原文摘要 · Abstract (English)
Harnessing low-light enhancement and domain adaptation, nighttime UAV tracking has made substantial strides. However, over-reliance on image enhancement, limited high-quality nighttime data, and a lack of integration between daytime and nighttime trackers hinder the development of an end-to-end trainable framework. Additionally, current ViT-based trackers demand heavy computational resources due to their reliance on the self-attention mechanism. In this paper, we propose a novel pure Mamba-based tracking framework (MambaNUT) that employs a state space model with linear complexity as its backbone, incorporating a single-stream architecture that integrates feature learning and template-search coupling within Vision Mamba. We introduce an adaptive curriculum learning (ACL) approach that dynamically adjusts sampling strategies and loss weights, thereby improving the model's ability of generalization. Our ACL is composed of two levels of curriculum schedulers: (1) sampling scheduler that transforms the data distribution from imbalanced to balanced, as well as from easier (daytime) to harder (nighttime) samples; (2) loss scheduler that dynamically assigns weights based on the size of the training set and IoU of individual instances. Exhaustive experiments on multiple nighttime UAV tracking benchmarks demonstrate that the proposed MambaNUT achieves state-of-the-art performance while requiring lower computational costs. The code will be available at https://github.com/wuyou3474/MambaNUT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。