用事件相机动态信号优化视频模型令牌剪枝，提升推理效率与精度。
EventPrune: Cascaded Event-Assisted Token Pruning for Efficient First-Person Dynamic Spatial Reasoning
- 基于事件相机高频运动信号，分三阶段动态筛选关键帧与视觉令牌。
- 减少80%视觉令牌后，准确率反超全量令牌基线，推理速度提升1.89倍。
- 适用于第一人称动态空间推理场景，尤其适合实时嵌入式系统部署。
第一人称动态空间推理需追踪连续运动与精确几何结构，但基于Transformer的视频大模型存在二次注意力开销，密集视觉令牌计算成本高。现有令牌剪枝方法多依赖离散静态快照，难以保留运动与几何线索。本文提出事件级联剪枝（ECP），据知是首个无需训练的框架，利用事件相机的高频运动信号作为连续运动先验，指导令牌选择。ECP包含三阶段：事件触发因果采样锚定运动信息关键帧，事件引导运动显著性过滤抑制无事件活动的视觉令牌，事件-注意力排序融合校准空间注意力。在80%视觉令牌减少下，ECP性能优于全令牌基线（37.62% vs. 36.31%），实现1.89倍推理加速与52% GFLOPs降低。我们还构建了首个真实世界RGB-事件基准ESR-Real，ECP在此上相较全令牌基线提升2.68个百分点准确率。
原文摘要 · Abstract (English)
First-person dynamic spatial reasoning requires models to track continuous motion and precise geometric structure, but the quadratic attention cost of Transformer-based Video-LLMs makes dense visual tokens computationally expensive. Existing token pruning paradigms predominantly rely on discrete static snapshots, failing to preserve the motion and geometric cues essential for reasoning. We propose Event Cascade Pruning (ECP), to our knowledge the first training-free framework that leverages the high-frequency motion cues from event cameras as a continuous event-guided motion prior to guide token selection. ECP combines three stages: Event-Triggered Causal Sampling to anchor motion-informative keyframes, Event-guided Motion Saliency Filtering to suppress event-inactive visual tokens, and Event-Attention Ranking Fusion to calibrate spatial attention with motion-salient dynamics. With 80% visual token reduction, ECP outperforms the full-token baseline (37.62% vs. 36.31%) while achieving 1.89x inference speedup and 52% GFLOPs reduction. We further introduce ESR-Real, the first real-world RGB-event benchmark for first-person spatial reasoning, where ECP improves accuracy by 2.68 percentage points over full-token baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。