arXiv:2608.04610cs.CV2026-08中稿 · ACM MM 2026

提出分层空间聚类压缩方法,高效减少3D视觉语言模型的冗余信息。

HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding

论文配图:HiSC: Hierarchical Spatial Clustering Token Compression for Efficient 3D Scene Understanding
图 1 · 摘自论文原文
  • 基于几何与语义联合聚类,将冗余视觉标记合并为结构化区块。
  • 在高压缩率下保持90%以上令牌削减,性能损失极小。
  • 无需训练,适合需快速推理的3D场景理解任务。

3D视觉语言模型(3D VLMs)虽能实现多视角场景的空间推理,但因重复观测和大量无信息区域导致显著的令牌冗余,计算开销大。尽管视觉令牌压缩在2D VLM中已显成效,却难以捕捉3D场景的结构特性,造成空间覆盖不全与细节丢失。本文提出无需训练的分层空间聚类压缩框架HiSC。HiSC将令牌压缩从单个令牌选择提升至簇级处理,利用几何与语义联合线索构建空间对齐的聚类。首先引入基于空间图的合并(SGraM)策略,将跨视角冗余建模为空间连通性,提前合并高度相似的冗余令牌;随后在大模型推理中采用空间聚类剪枝(SCluP)范式,实现簇间与簇内分层压缩,在保持物体实例完整性的同时保留关键区域的细粒度细节。在多个3D推理基准上的实验验证了其有效性,尤其在高视觉令牌压缩率下表现优异。此外,HiSC可在超过90%的令牌压缩率下维持最小性能下降。代码已开源:https://github.com/elecreak/HiSC。

原文摘要 · Abstract (English)

3D vision-language models (3D VLMs) enable spatial reasoning over multi-view scenes but suffer from substantial token redundancy due to duplicated observations and large uninformative regions, leading to high computational cost. Although visual token compression has shown promise in accelerating 2D VLMs, it fails to capture the structured nature of 3D scenes and leads to incomplete spatial coverage and loss of fine-grained details. In this paper, we propose \textbf{HiSC}, a training-free framework for hierarchical spatial clustering token compression in 3D VLMs. HiSC lifts token compression from token-level selection to cluster-level processing by organizing tokens into spatially grounded clusters using joint geometric and semantic cues. Specifically, we first introduce a \textbf{spatial graph-based merging (SGraM) strategy} that models cross-view redundancy as spatial connectivity and consolidates physically consistent regions, effectively merging extremely similar redundant tokens prior to LLM inference. We then propose a \textbf{spatial clustering-based pruning (SCluP) paradigm} within LLM inference, which performs hierarchical compression across clusters and within clusters, preserving object instance completeness while retaining fine-grained details for important regions. Extensive experiments on diverse 3D reasoning benchmarks show validate the effectiveness of HiSC, particularly under high visual token pruning ratios. Besides, HiSC achieves over 90\% token reduction with minimal performance degradation. Code is accessible at https://github.com/elecreak/HiSC.

3D视觉令牌压缩空间聚类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。