3D点云变换器可大幅减少令牌数量，提升效率。
How Many Tokens Do 3D Point Cloud Transformer Architectures Really Need?
- 提出全局图合并方法，自动压缩冗余令牌。
- 令牌减少90%-95%后性能仍保持竞争力。
- 适合追求高效3D视觉模型的开发者使用。
近年来，3D点云变换器在语义分割和重建等任务中取得了最先进成果。然而，这些模型通常依赖密集的令牌表示，导致训练和推理时计算与内存开销巨大。本文发现令牌存在显著冗余，造成严重效率损失。我们提出gitmerge3D，一种全局感知的图令牌合并方法，可在保持竞争性性能的同时将令牌数量减少90%-95%。这一发现挑战了“更多令牌带来更好性能”的普遍假设，表明当前许多模型存在过度令牌化且缺乏可扩展性优化。我们在多个3D视觉任务上验证该方法，均实现计算效率的持续提升。本工作是首个系统评估大规模3D变换器冗余的研究，为构建更高效的3D基础架构提供关键洞察。代码与模型权重已公开于https://gitmerge3d.github.io。
原文摘要 · Abstract (English)
Recent advances in 3D point cloud transformers have led to state-of-the-art results in tasks such as semantic segmentation and reconstruction. However, these models typically rely on dense token representations, incurring high computational and memory costs during training and inference. In this work, we present the finding that tokens are remarkably redundant, leading to substantial inefficiency. We introduce gitmerge3D, a globally informed graph token merging method that can reduce the token count by up to 90-95% while maintaining competitive performance. This finding challenges the prevailing assumption that more tokens inherently yield better performance and highlights that many current models are over-tokenized and under-optimized for scalability. We validate our method across multiple 3D vision tasks and show consistent improvements in computational efficiency. This work is the first to assess redundancy in large-scale 3D transformer models, providing insights into the development of more efficient 3D foundation architectures. Our code and checkpoints are publicly available at https://gitmerge3d.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。