通过频域适配器实现点云模型轻量微调，大幅降低参数量与计算开销。
Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning
- 将点云从空间域转换到频域，用正交基分离冗余信息
- 仅需0.3%参数量即可达到全量微调性能，提升效率97%以上
- 适合资源受限场景下的点云任务快速部署
近期，利用预训练技术提升点云模型表现成为研究热点。然而，现有方法通常需要对预训练模型进行全量微调才能在下游任务上取得良好效果，这带来高存储和计算成本。为此，本文提出一种新型参数高效微调方法PointGST（Point cloud Graph Spectral Tuning），冻结预训练模型并引入轻量级可训练的点云频域适配器（PCSA），在频域中进行参数微调。核心思想基于两点观察：1）冻结模型内部的点特征在空间域可能存在混淆；2）任务特异性内在信息对知识迁移至关重要。PointGST将点云特征从空间域转换至频域，利用正交分量有效解耦特征间的混淆。同时生成的频域基底包含下游点云的内在结构信息，支持更精准的调参。实验表明，PointGST在多个挑战性点云数据集上超越全量微调方法，且可训练参数减少超过97%，显著降低训练成本，是高效的点云学习解决方案。代码将开源于https://github.com/jerryfeng2003/PointGST。
原文摘要 · Abstract (English)
Recently, leveraging pre-training techniques to enhance point cloud models has become a prominent research topic. However, existing approaches typically require full fine-tuning of pre-trained models to achieve satisfactory performance on downstream tasks, which is storage-intensive and computationally demanding. To address this issue, we propose a novel Parameter-Efficient Fine-Tuning (PEFT) method for point cloud, called \textbf{PointGST} (\textbf{Point} cloud \textbf{G}raph \textbf{S}pectral \textbf{T}uning). PointGST freezes the pre-trained model and introduces a lightweight, trainable Point Cloud Spectral Adapter (PCSA) for fine-tuning parameters in the spectral domain. The core idea is built on two observations: 1) The inner tokens from frozen models might present confusion in the spatial domain; 2) Task-specific intrinsic information is important for transferring the general knowledge to the downstream task. Specifically, PointGST transfers the point tokens from the spatial domain to the spectral domain, effectively de-correlating confusion among tokens by using orthogonal components for separation. Moreover, the generated spectral basis involves intrinsic information about the downstream point clouds, enabling more targeted tuning. As a result, PointGST facilitates the efficient transfer of general knowledge to downstream tasks while significantly reducing training costs. Extensive experiments on challenging point cloud datasets across various tasks demonstrate that PointGST not only outperforms its fully fine-tuning counterpart but also significantly reduces trainable parameters, making it a promising solution for efficient point cloud learning. The code will be made available at https://github.com/jerryfeng2003/PointGST
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。