不训练即可大幅压缩视觉语言模型计算量,提升自动驾驶效率。
ST-Prune: Training-Free Spatio-Temporal Token Pruning for Vision-Language Models in Autonomous Driving

- 通过运动感知和环视结构,同时剪除视频时空冗余信息。
- 在90%剪枝率下仍保持接近全模型性能,部分指标更优。
- 无需训练、可直接部署,适合实时自动驾驶系统使用。
视觉语言模型(VLMs)已成为自动驾驶系统的核心,但多视角摄像头与多帧视频输入带来的巨大计算开销严重制约其部署。现有剪枝方法主要针对单图输入,孤立处理每帧或视角,无法利用驾驶场景中的时空冗余。为此,我们提出ST-Prune,一种无需训练、即插即用的框架,包含两个互补模块:运动感知时间剪枝(MTP)和环视空间剪枝(RSP)。MTP通过编码运动剧烈度与时间新近性作为软约束,在多样性选择中优先保留动态轨迹与当前帧内容;RSP则利用环视相机几何结构,惩罚双向跨视角相似性,消除重复投影与残余背景。二者协同完成完整的时空剪枝过程,在严格压缩下保留关键场景信息。在四个涵盖感知、预测与规划的基准上验证,ST-Prune成为训练自由剪枝的新标杆。即使在90%令牌减少时,仍实现近乎无损性能,某些指标超越完整模型,且推理速度与现有剪枝方法相当。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) have become central to autonomous driving systems, yet their deployment is severely bottlenecked by the massive computational overhead of multi-view camera and multi-frame video input. Existing token pruning methods, primarily designed for single-image inputs, treat each frame or view in isolation and thus fail to exploit the inherent spatio-temporal redundancies in driving scenarios. To bridge this gap, we propose ST-Prune, a training-free, plug-and-play framework comprising two complementary modules: Motion-aware Temporal Pruning (MTP) and Ring-view Spatial Pruning (RSP). MTP addresses temporal redundancy by encoding motion volatility and temporal recency as soft constraints within the diversity selection objective, prioritizing dynamic trajectories and current-frame content over static historical background. RSP further resolves spatial redundancy by exploiting the ring-view camera geometry to penalize bilateral cross-view similarity, eliminating duplicate projections and residual background that temporal pruning alone cannot suppress. These two modules together constitute a complete spatio-temporal pruning process, preserving key scene information under strict compression. Validated across four benchmarks spanning perception, prediction, and planning, ST-Prune establishes new state-of-the-art for training-free token pruning. Notably, even at 90\% token reduction, ST-Prune achieves near-lossless performance with certain metrics surpassing the full-model baseline, while maintaining inference speeds comparable to existing pruning approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。