用事件相机引导,动态删减无信息特征,提升检测效率与精度
Focus Through Motion: RGB-Event Collaborative Token Sparsification for Efficient Object Detection
- 基于事件相机感知变化,自适应剔除图像和事件数据中的低信息区域
- 在DSEC-Det和PKU-DAVIS-SOD上实现更高准确率与更低计算开销
- 适合需要高效实时目标检测的视觉系统,如自动驾驶、机器人导航
现有RGB-事件检测方法在特征提取与融合时对两类模态的低信息区域(图像背景、事件非活跃区)进行均匀处理,导致计算冗余和性能下降。尽管已有针对图像和事件模态的独立稀疏化方法,但其采用固定数量或阈值选择令牌,难以适应复杂度不同的样本。为此,我们提出FocusMamba,通过自适应协同稀疏化多模态特征并高效融合互补信息,实现精度与效率的更好平衡。具体地,设计了事件引导的多模态稀疏化(EGMS)策略,利用事件相机感知的场景变化,识别并自适应丢弃各模态中的低信息区域;基于稀疏化结果,提出跨模态聚焦融合(CMFF)模块,有效捕捉并整合双模态互补特征。在DSEC-Det和PKU-DAVIS-SOD数据集上的实验表明,所提方法在准确率和效率方面均优于现有方法。
原文摘要 · Abstract (English)
Existing RGB-Event detection methods process the low-information regions of both modalities (background in images and non-event regions in event data) uniformly during feature extraction and fusion, resulting in high computational costs and suboptimal performance. To mitigate the computational redundancy during feature extraction, researchers have respectively proposed token sparsification methods for the image and event modalities. However, these methods employ a fixed number or threshold for token selection, hindering the retention of informative tokens for samples with varying complexity. To achieve a better balance between accuracy and efficiency, we propose FocusMamba, which performs adaptive collaborative sparsification of multimodal features and efficiently integrates complementary information. Specifically, an Event-Guided Multimodal Sparsification (EGMS) strategy is designed to identify and adaptively discard low-information regions within each modality by leveraging scene content changes perceived by the event camera. Based on the sparsification results, a Cross-Modality Focus Fusion (CMFF) module is proposed to effectively capture and integrate complementary features from both modalities. Experiments on the DSEC-Det and PKU-DAVIS-SOD datasets demonstrate that the proposed method achieves superior performance in both accuracy and efficiency compared to existing methods. The code will be available at https://github.com/Zizzzzzzz/FocusMamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。