arXiv:2604.18260cs.CV2026-04中稿 · CVPR被引 1

通过几何引导剪枝,大幅减少3D视觉令牌数量而保持性能

Geometry-Guided 3D Visual Token Pruning for Video-Language Models

论文配图:Geometry-Guided 3D Visual Token Pruning for Video-Language Models
图 1 · 摘自论文原文
  • 基于几何信息建模跨帧相关性,分两阶段剪枝
  • 剪掉90%视觉令牌,性能保留超90%
  • 适合需要高效3D场景理解的模型部署

多模态大语言模型在2D视觉上表现卓越,促使研究向3D场景理解延伸。近期工作将3D场景表示为包含深度与相机位姿信息的3D空间视频,使预训练视频-语言模型可执行3D推理任务。然而,空间视频中庞大的视觉令牌数量仍是高效推理与上下文管理的主要瓶颈。现有剪枝方法忽略空间视频的视角一致性与剩余令牌的空间多样性,难以有效消除帧间冗余并保持场景完整性。本文提出Geo3DPruner,一种几何引导的3D视觉令牌剪枝框架。该方法首先通过几何感知全局注意力建模跨帧相关性,再执行两阶段剪枝:在体素内阶段选择每个体素内的代表性多视角特征;在体素间阶段通过选取全局分布的体素子集来保留空间多样性。在多个3D场景理解基准上的实验表明,Geo3DPruner在剪除90%视觉令牌的同时,仍能保持超过90%的原始性能,显著优于现有的文本引导与视觉引导剪枝方法。

原文摘要 · Abstract (English)

Multimodal large language models have demonstrated remarkable capabilities in 2D vision, motivating their extension to 3D scene understanding. Recent studies represent 3D scenes as 3D spatial videos composed of image sequences with depth and camera pose information, enabling pre-trained video-language models to perform 3D reasoning tasks. However, the large number of visual tokens in spatial videos remains a major bottleneck for efficient inference and context management. Existing pruning methods overlook the view consistency of spatial videos and the spatial diversity of the remaining tokens, which prevents them from effectively removing inter-frame redundancy and preserving scene completeness. In this paper, we propose Geo3DPruner, a Geometry-Guided 3D Visual Token Pruning framework. Geo3DPruner first models cross-frame relevance through geometry-aware global attention, and then performs a two-stage pruning process. The intra-voxel stage selects representative multi-view features within each voxel, while the inter-voxel stage preserves spatial diversity by selecting a globally distributed subset of voxels. Extensive experiments on multiple 3D scene understanding benchmarks demonstrate that Geo3DPruner retains over 90% of the original performance while pruning 90% of visual tokens, significantly outperforming existing text-guided and vision-guided pruning methods.

3D视觉模型剪枝视频语言模型几何引导

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。