针对视觉目标跟踪中多模态信号退化问题，提出可主动扰动的关联记忆检索框架。
Active Adversarial Perturbation-driven Associative Memory Retrieval for RGB-Event Visual Object Tracking
- 设计双层级对抗扰动模拟信号缺失与局部目标丢失
- 通过足迹引导的霍普菲尔德检索实现可控历史信息补偿
- 在多个数据集上显著提升复杂环境下跟踪鲁棒性
RGB-Event跟踪通过融合RGB外观纹理与事件传感器的密集时间运动线索，提升了定位鲁棒性。然而，真实场景中存在多种结构化信号退化问题，导致传统多模态融合失效。在恶劣环境中，任一模态均可能严重失真，目标常因遮挡、边缘截断或前景杂乱而呈现不完整状态。为此，本文提出面向鲁棒性缺失目标与模态退化的分层扰动与检索框架APRTrack。为模拟真实信号损坏，该框架在模态与空间两个层级构建对抗扰动分支，分别模拟全模态失效与局部目标区域缺失。设计分层路由机制解耦两类扰动训练流程，有效避免叠加退化约束引发的特征坍塌。进一步提出基于足迹引导的通道校准霍普菲尔德检索（FCHR）模块，通过查询与记忆库间关联足迹评估检索置信度，并在霍普菲尔德匹配前校准度量空间，实现限定于目标区域的可控历史特征补偿。在FE108、COESOT、VisEvent和FELT数据集上的大量实验验证了所提策略在RGB-Event视觉目标跟踪中的有效性。源代码与预训练模型将发布于https://github.com/Event-AHU/OpenEvTracking。
原文摘要 · Abstract (English)
RGB-Event tracking improves localization robustness by fusing RGB appearance textures and dense temporal motion cues from event sensors. While this multi-modal scheme broadens tracking applicability, real-world scenes suffer diverse structured signal degradations that hinder traditional multi-modal fusion. In harsh environments, either modality can lose reliability drastically, and targets frequently appear incomplete due to occlusion, edge truncation and foreground clutter.To tackle the above challenges, we present a hierarchical perturbation and retrieval framework tailored for RGB-Event tracking with robustness against partial target missing and modal degradation, termed APRTrack. To mimic real-world signal corruption, APRTrack constructs structured degradation via two adversarial perturbation branches at the modality and spatial levels, which separately simulate full-modal failure and localized target region absence. A hierarchical routing mechanism is designed to disentangle the training pipelines of the two perturbation types, effectively eliminating feature collapse induced by superimposed degradation constraints. Furthermore, we devise Footprint-guided Channel-calibrated Hopfield Retrieval (FCHR) for reliable historical information compensation. This module evaluates retrieval confidence based on association footprints between queries and memory banks, and calibrates the retrieval metric space prior to Hopfield matching, realizing controllable historical feature compensation bounded to target regions. Extensive experiments on FE108, COESOT, VisEvent, and FELT datasets demonstrate the effectiveness of our proposed strategies for the RGB-Event visual object tracking. The source code and pre-trained models will be released on https://github.com/Event-AHU/OpenEvTracking
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。