arXiv:2505.22444cs.CV2025-05NeurIPS被引 2

提出GEM模块,让3D点云模型用1.6%参数就能高效适配新任务。

On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentation

  • 设计几何感知的轻量模块,融合局部位置编码与注意力机制。
  • 仅更新1.6%参数即达全量微调性能,训练耗时和内存大幅降低。
  • 适合需要快速部署3D场景分割模型的研究者和工程师。

大规模预训练点云模型显著推进了3D场景理解,但将其适配到下游任务通常需全量微调,带来高昂计算与存储成本。现有参数高效微调(PEFT)方法在3D点云模型上表现不佳,因忽视了显著的几何与空间分布差异。传统PEFT常将点视为无序令牌,忽略了3D建模中的局部空间结构与全局几何上下文。为此,我们提出几何编码混合器(GEM),一种专为3D点云变换器设计的几何感知PEFT模块。GEM显式融合细粒度局部位置编码与轻量级潜在注意力机制,有效捕捉全面的全局上下文,缓解空间与几何分布不匹配问题。大量实验表明,GEM性能可媲美甚至超越全量微调,仅更新模型1.6%参数,优于其他PEFT方法。训练时间与内存需求显著减少,为大规模3D点云模型的高效、可扩展、几何感知微调树立新基准。代码见https://github.com/LiyaoTang/GEM。

原文摘要 · Abstract (English)

The emergence of large-scale pre-trained point cloud models has significantly advanced 3D scene understanding, but adapting these models to specific downstream tasks typically demands full fine-tuning, incurring high computational and storage costs. Parameter-efficient fine-tuning (PEFT) techniques, successful in natural language processing and 2D vision tasks, would underperform when naively applied to 3D point cloud models due to significant geometric and spatial distribution shifts. Existing PEFT methods commonly treat points as orderless tokens, neglecting important local spatial structures and global geometric contexts in 3D modeling. To bridge this gap, we introduce the Geometric Encoding Mixer (GEM), a novel geometry-aware PEFT module specifically designed for 3D point cloud transformers. GEM explicitly integrates fine-grained local positional encodings with a lightweight latent attention mechanism to capture comprehensive global context, thereby effectively addressing the spatial and geometric distribution mismatch. Extensive experiments demonstrate that GEM achieves performance comparable to or sometimes even exceeding full fine-tuning, while only updating 1.6% of the model's parameters, fewer than other PEFT methods. With significantly reduced training time and memory requirements, our approach thus sets a new benchmark for efficient, scalable, and geometry-aware fine-tuning of large-scale 3D point cloud models. Code is available at https://github.com/LiyaoTang/GEM.

3D分割参数高效点云

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。