用几何提示提升3D视觉模型效率,仅用2%参数达全微调效果
GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
- 设计点提示与点位移提示,增强模型对几何细节的感知
- 在多个基准上性能超越主流参数高效微调方法,仅需2.19%可训练参数
- 适合需要高效适配3D点云任务的研究者和工业应用
预训练3D视觉模型在点云数据上表现出色,但全量微调成本高。现有参数高效微调方法多依赖输入标记提示,难以捕捉点云固有的几何信息。为此,我们提出几何感知点云提示(GAPrompt),通过引入点提示作为辅助输入,显式引导模型关注细粒度几何特征;设计点位移提示器,从点云中提取全局形状信息,实现实例级几何调整;并提出提示传播机制,将形状信息融入模型特征提取过程,强化几何表征能力。大量实验表明,GAPrompt显著优于当前最优参数高效微调方法,在多个基准上表现接近全微调,仅使用2.19%的可训练参数。代码已公开于https://github.com/zhoujiahuan1991/ICML2025-GAPrompt。
原文摘要 · Abstract (English)
Pre-trained 3D vision models have gained significant attention for their promising performance on point cloud data. However, fully fine-tuning these models for downstream tasks is computationally expensive and storage-intensive. Existing parameter-efficient fine-tuning (PEFT) approaches, which focus primarily on input token prompting, struggle to achieve competitive performance due to their limited ability to capture the geometric information inherent in point clouds. To address this challenge, we propose a novel Geometry-Aware Point Cloud Prompt (GAPrompt) that leverages geometric cues to enhance the adaptability of 3D vision models. First, we introduce a Point Prompt that serves as an auxiliary input alongside the original point cloud, explicitly guiding the model to capture fine-grained geometric details. Additionally, we present a Point Shift Prompter designed to extract global shape information from the point cloud, enabling instance-specific geometric adjustments at the input level. Moreover, our proposed Prompt Propagation mechanism incorporates the shape information into the model's feature extraction process, further strengthening its ability to capture essential geometric characteristics. Extensive experiments demonstrate that GAPrompt significantly outperforms state-of-the-art PEFT methods and achieves competitive results compared to full fine-tuning on various benchmarks, while utilizing only 2.19% of trainable parameters. Our code is available at https://github.com/zhoujiahuan1991/ICML2025-GAPrompt.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。