提出Prompt调控框架,让3D点云模型在提升识别能力的同时保持强泛化性。
Point-PRC: A Prompt Learning Based Regulation Framework for Generalizable Point Cloud Analysis
- 通过显式约束提示学习轨迹,促进任务特异性与通用知识的协同
- 在多个3DDG基准上同时提升识别准确率与跨域泛化能力
- 适合作为插件嵌入现有大模型,推动3D领域泛化研究
本文研究基于提示学习的大规模3D模型在3D域泛化(3DDG)方面的表现。尽管参数高效提示微调能显著提升下游任务性能,但会严重损害3D域泛化能力。为此,我们提出一种综合性调控框架,使可学习提示主动与大型3D模型中已习得的通用知识交互,以维持良好泛化性。具体地,通过最大化任务特定预测与任务无关知识之间的互一致性,对提示学习轨迹施加多重显式约束。该框架设计为即插即用模块,可嵌入主流大型3D模型。实验表明,本方法不仅持续提升泛化能力,还在多个3DDG基准上显著增强任务特定识别性能。针对3DDG研究与评估的缺失,我们构建了三个新基准:base-to-new、cross-dataset和few-shot generalization benchmark,以丰富该领域并激励未来研究。代码与基准数据集见:https://github.com/auniquesun/Point-PRC。
原文摘要 · Abstract (English)
This paper investigates the 3D domain generalization (3DDG) ability of large 3D models based on prevalent prompt learning. Recent works demonstrate the performances of 3D point cloud recognition can be boosted remarkably by parameter-efficient prompt tuning. However, we observe that the improvement on downstream tasks comes at the expense of a severe drop in 3D domain generalization. To resolve this challenge, we present a comprehensive regulation framework that allows the learnable prompts to actively interact with the well-learned general knowledge in large 3D models to maintain good generalization. Specifically, the proposed framework imposes multiple explicit constraints on the prompt learning trajectory by maximizing the mutual agreement between task-specific predictions and task-agnostic knowledge. We design the regulation framework as a plug-and-play module to embed into existing representative large 3D models. Surprisingly, our method not only realizes consistently increasing generalization ability but also enhances task-specific 3D recognition performances across various 3DDG benchmarks by a clear margin. Considering the lack of study and evaluation on 3DDG, we also create three new benchmarks, namely base-to-new, cross-dataset and few-shot generalization benchmarks, to enrich the field and inspire future research. Code and benchmarks are available at \url{https://github.com/auniquesun/Point-PRC}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。