揭示视频检测器是否真用时间信息，提出可测方法与高效模型
Temporal Preservation over Processing: Diagnosing and Designing Spatiotemporal Single-Stage Video Detectors
- 设计扰动测试框架，检验模型是否依赖跨时序信息
- 发现2D模型丢帧即崩溃，而时空模型能靠前帧恢复预测
- 构建实时新模型，保留时间深度使性能提升3.7个百分点
单阶段视频目标检测器在时间敏感应用中日益普及，但其是否真正利用时序上下文仍不明确——标准指标会奖励正确预测，掩盖了模型是否仅依赖单一关键帧。我们从两个方向解决此问题：首先提出TemporalLens，一种模型无关的诊断框架，通过控制扰动、结构遮挡、时间打乱、冗余注入和分辨率退化等手段，揭示检测器是否实际使用跨时间信息。应用于堆叠帧2D检测器与我们的YOLO-3D架构，结果表明：当目标帧被移除时，2D模型性能骤降，而时空模型能从早期帧恢复预测，表现出真正的时序依赖。其次，我们设计了基于YOLOv8的模块化实时时空检测器YOLO-3D，证明仅在主干网络中保留时间深度即可显著提升性能，在32帧平均下实现+3.7个百分点的mAP@50提升。诊断与架构协同，将“是否推理时间”转化为可测量、可行动的问题。
原文摘要 · Abstract (English)
Single-stage video object detectors are increasingly deployed in time-critical applications, yet it remains unclear whether these models genuinely reason over temporal context or merely exploit a single informative frame-a gap hidden by standard metrics, which reward correct predictions regardless of how they are reached. We address this from two complementary directions: first, we propose TemporalLens, a model-agnostic diagnostic framework probing temporal dependence through controlled perturbations, structured occlusions, temporal shuffling, redundancy injection, and resolution degradation, revealing whether a detector actually uses information across time. Applied to stacked-frame 2D detectors and our YOLO-3D architecture, it exposes behavioural differences invisible to mAP: stacked 2D models collapse when the target frame is removed, while spatiotemporal models recover predictions from earlier frames, a signature of real temporal reliance. Second, we detail YOLO-3D, a modular real-time spatiotemporal detector built on YOLOv8, and show that simply preserving temporal depth through the backbone is the dominant performance driver (+3.7 pp mAP@50 at 32 frames averaged across scales). Together, the diagnostics and architecture turn "does this detector reason over time?" into a measurable, actionable question.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。