用低秩+关键点选择,让点云模型少调参数也能高效微调。
PointLoRA: Low-Rank Adaptation with Token Selection for Point Cloud Learning
- 在点云变换器中嵌入低秩适配层,减少可训练参数
- 多尺度点选机制提取局部关键信息,补全全局特征
- 仅需3.43%参数量,适合算力有限的场景
点云自监督表示学习在提升预训练模型性能方面已证明有效。然而,随着预训练模型日益复杂,完全微调其参数对计算与存储资源需求巨大。参数高效微调(PEFT)方法能缓解此问题,但多数现有方法依赖复杂的适配器或提示机制,反而增加了可调参数。本文提出PointLoRA,结合低秩适配(LoRA)与多尺度点选择,高效微调点云模型。该方法将LoRA层嵌入点云变换器中最耗参数的部分,降低可训练参数需求,同时增强全局特征捕捉能力;多尺度点选择则提取关键局部信息作为下游微调提示,有效补充LoRA所捕获的全局上下文。在多个预训练模型及三个公开挑战性数据集上的实验表明,本方法仅需3.43%可训练参数即可达到竞争力性能,适用于资源受限的应用。源代码见:https://github.com/songw-zju/PointLoRA。
原文摘要 · Abstract (English)
Self-supervised representation learning for point cloud has demonstrated effectiveness in improving pre-trained model performance across diverse tasks. However, as pre-trained models grow in complexity, fully fine-tuning them for downstream applications demands substantial computational and storage resources. Parameter-efficient fine-tuning (PEFT) methods offer a promising solution to mitigate these resource requirements, yet most current approaches rely on complex adapter and prompt mechanisms that increase tunable parameters. In this paper, we propose PointLoRA, a simple yet effective method that combines low-rank adaptation (LoRA) with multi-scale token selection to efficiently fine-tune point cloud models. Our approach embeds LoRA layers within the most parameter-intensive components of point cloud transformers, reducing the need for tunable parameters while enhancing global feature capture. Additionally, multi-scale token selection extracts critical local information to serve as prompts for downstream fine-tuning, effectively complementing the global context captured by LoRA. The experimental results across various pre-trained models and three challenging public datasets demonstrate that our approach achieves competitive performance with only 3.43% of the trainable parameters, making it highly effective for resource-constrained applications. Source code is available at: https://github.com/songw-zju/PointLoRA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。