arXiv:2501.02467cs.CV2025-01NeurIPS被引 14

用噪声还原思路提升目标跟踪鲁棒性,速度不降反而更快。

DeTrack: In-model Latent Denoising Learning for Visual Object Tracking

  • 将跟踪任务转为模型内潜空间的去噪学习,利用噪声框训练增强泛化能力。
  • 在多个基准数据集上达到领先性能,尤其在遮挡和形变场景下表现优异。
  • 适合追求高鲁棒性且对实时性有要求的目标跟踪应用。

以往的视觉目标跟踪方法采用图像特征回归或坐标自回归模型进行边界框预测。前者依赖匹配结果且忽略位置先验,后者仅能基于训练集中可见的边界框训练,导致测试时面对未见数据性能下降。受扩散模型启发,我们引入噪声到边界框中生成带噪框用于训练,从而提升模型对未知数据的鲁棒性。本文提出一种新范式:将视觉目标跟踪问题建模为去噪学习过程。然而,跟踪通常需实时运行,直接应用扩散模型会严重拖慢速度。因此,我们把去噪过程分解为模型内的多个去噪块,而非多次迭代运行,形成‘模型内潜空间去噪学习’。具体地,提出一种由多个去噪块组成的去噪视觉变换器(Denoising ViT)。每个去噪块将模板和搜索区域嵌入作为条件,负责去除预测框中的噪声,多层堆叠协同完成整体去噪。随后,结合图像特征与轨迹信息优化去噪后的边界框,并引入轨迹记忆和视觉记忆以增强跟踪稳定性。实验验证了该方法的有效性,在多个挑战性数据集上取得具有竞争力的结果。

原文摘要 · Abstract (English)

Previous visual object tracking methods employ image-feature regression models or coordinate autoregression models for bounding box prediction. Image-feature regression methods heavily depend on matching results and do not utilize positional prior, while the autoregressive approach can only be trained using bounding boxes available in the training set, potentially resulting in suboptimal performance during testing with unseen data. Inspired by the diffusion model, denoising learning enhances the model's robustness to unseen data. Therefore, We introduce noise to bounding boxes, generating noisy boxes for training, thus enhancing model robustness on testing data. We propose a new paradigm to formulate the visual object tracking problem as a denoising learning process. However, tracking algorithms are usually asked to run in real-time, directly applying the diffusion model to object tracking would severely impair tracking speed. Therefore, we decompose the denoising learning process into every denoising block within a model, not by running the model multiple times, and thus we summarize the proposed paradigm as an in-model latent denoising learning process. Specifically, we propose a denoising Vision Transformer (ViT), which is composed of multiple denoising blocks. In the denoising block, template and search embeddings are projected into every denoising block as conditions. A denoising block is responsible for removing the noise in a predicted bounding box, and multiple stacked denoising blocks cooperate to accomplish the whole denoising process. Subsequently, we utilize image features and trajectory information to refine the denoised bounding box. Besides, we also utilize trajectory memory and visual memory to improve tracking stability. Experimental results validate the effectiveness of our approach, achieving competitive performance on several challenging datasets.

目标跟踪去噪学习ViT实时性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。