提出无需训练的视频时间定位剪枝方法，保留关键证据链提升效率
Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Grounding
- 按帧分配预算，分三类选关键帧：对象、运动、上下文token
- 仅用12.5%视觉标记即保持95.4% mIoU，预填充速度提升5.8倍
- 适合追求高效视频理解且需精准边界定位的应用场景
视频时间定位（VTG）旨在长而未剪辑的视频中定位与查询相关的时段，但视频-语言模型成本高昂。尽管近期无训练的标记剪枝在视频问答中表现良好，但直接应用于VTG会导致性能大幅下降，因VTG依赖边界敏感的证据和跨帧推理链。本文提出两个针对VTG的剪枝原则：证据保留（重点保留事件边界附近的查询关键区域）和连通性强度（保持跨帧连接以实现长程证据聚合）。基于此，提出无训练剪枝框架SemVID，构建具有互补语义角色的紧凑且连贯的标记子集。SemVID首先通过平衡查询相关性与帧间差异分配每帧预算，避免过度剪枝；随后选择三类标记：用于多样化查询关键证据的对象标记、捕捉有意义变化并作为跨帧传递媒介的运动标记、维持场景连续性的上下文标记。大量实验表明，SemVID在极低标记比例下仍保持高精度，仅使用12.5%视觉标记即可达到95.4% mIoU，并实现最高5.8倍的预填充加速，显著优于现有方法。
原文摘要 · Abstract (English)
Video Temporal Grounding (VTG) localizes the temporal boundaries of query-relevant moments in long, untrimmed videos, making video-language-model prohibitively expensive. While recent training-free token pruning has shown success in video question answering, naively applying these objectives to VTG causes drastic degradation, as VTG crucially depends on boundary-sensitive evidence and cross-frame reasoning chains. We therefore identify two VTG-specific pruning principles: evidence retention, which keeps query-critical patches especially around event boundaries, and connectivity strength, which preserves cross-frame connectivity for long-range evidence aggregation. Building on these insights, we propose SemVID, a training-free pruning framework that constructs a compact yet coherent token subset with complementary semantic roles. SemVID first allocates per-frame budgets by balancing query relevance and inter-frame variation to avoid over-pruned segments, and then selects three types of tokens: object tokens for diverse query-critical evidence, motion tokens to capture meaningful transitions and serve as cross-frame relays, and context tokens for scene continuity. Extensive experiments show that SemVID achieves a strong accuracy-efficiency trade-off, retaining up to 95.4% mIoU with only 12.5% visual tokens and delivering up to a 5.8x prefill speedup, consistently outperforming prior methods under the same budgets. Our code is available at https://github.com/JiaqiLi404/SemVID
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。