用实例掩码增强视频目标检测，提升准确率与速度
Beyond Boxes: Mask-Guided Spatio-Temporal Feature Aggregation for Video Object Detection
- 基于实例掩码提取特征，避免背景干扰
- 在ImageNet VID上达87.9% mAP，33 FPS实时运行
- 方法通用性强，适合多目标追踪等视频任务
视频目标检测的核心挑战在于有效利用时序信息以增强目标表征。传统方法如区域提议聚合常因引入背景信息导致特征方差。本文提出FAIM，通过实例掩码引导的特征聚合机制，显著提升对目标动态的理解。设计轻量级实例特征提取模块（IFEM）学习实例掩码特征，并构建时序实例分类聚合模块（TICAM），跨帧聚合掩码与分类特征。以YOLOX为基础检测器，FAIM在ImageNet VID数据集上达到87.9% mAP，单张2080Ti GPU下实现33 FPS，刷新速度-精度平衡基准。多数据集实验验证该方法具有鲁棒性、方法无关性，在多目标追踪中表现优异，适用于更广泛的视频理解任务。
原文摘要 · Abstract (English)
The primary challenge in Video Object Detection (VOD) is effectively exploiting temporal information to enhance object representations. Traditional strategies, such as aggregating region proposals, often suffer from feature variance due to the inclusion of background information. We introduce a novel instance mask-based feature aggregation approach, significantly refining this process and deepening the understanding of object dynamics across video frames. We present FAIM, a new VOD method that enhances temporal Feature Aggregation by leveraging Instance Mask features. In particular, we propose the lightweight Instance Feature Extraction Module (IFEM) to learn instance mask features and the Temporal Instance Classification Aggregation Module (TICAM) to aggregate instance mask and classification features across video frames. Using YOLOX as a base detector, FAIM achieves 87.9% mAP on the ImageNet VID dataset at 33 FPS on a single 2080Ti GPU, setting a new benchmark for the speed-accuracy trade-off. Additional experiments on multiple datasets validate that our approach is robust, method-agnostic, and effective in multi-object tracking, demonstrating its broader applicability to video understanding tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。