arXiv:2606.24175cs.CV2026-06被引 6

PoinTriE用自监督训练+轻量微调,让点云视频模型更省资源、更高效。

Tri-Efficient Transfer Learning for Point Cloud Videos

论文配图:Tri-Efficient Transfer Learning for Point Cloud Videos
图 1 · 摘自论文原文
  • 自动生成运动轨迹,结合图文与2D投影做多模态自监督
  • 微调仅更新轻量侧网络,内存占用降低90%以上
  • 适合资源有限但需高精度点云视频理解的场景

尽管点云基础模型显著提升了点云视频理解能力,现有参数高效微调(PEFT)方法仍面临两大瓶颈:大规模点云数据集标注成本高昂,以及严重的内存瓶颈。本文旨在从已有数据中挖掘更丰富的监督信号,而非盲目扩大数据规模。核心原则是微调的内存开销必须远低于全量微调,而这一目标当前的PEFT技术仍未实现。为此,我们提出三个关键需求:数据、参数与内存效率,并构建统一框架PoinTriE,在三方面均表现优异。预训练阶段,通过刚性变换合成伪运动轨迹,配以文本语料和从原始点云生成的2D投影;提出几何-运动对偶网络,通过多模态对比学习、刚性旋转预测及运动分布差异优化,生成密集自监督信号。微调时冻结预训练主干,仅更新基于LoRA构建的轻量级时空侧网络,并引入梯度流掩码策略,同时降低内存与参数开销。大量实验表明,PoinTriE在动作识别与语义分割任务上达到新最优性能。

原文摘要 · Abstract (English)

While point cloud foundation models have significantly advanced point cloud video understanding, existing parameter-efficient fine-tuning (PEFT) methods still suffer from two critical limitations: prohibitive annotation costs for large-scale point cloud datasets and severe memory bottlenecks. In this paper, we aim to mine richer supervision signals from existing data rather than blindly scaling datasets. A further key principle is that the memory footprint of fine-tuning must be drastically reduced compared to full fine-tuning, which remains elusive for current PEFT techniques. Driven by these challenges, we identify three core desiderata: data-, parameter-, and memory efficiency, and present PoinTriE, a unified framework that excels along all three dimensions. For pre-training, pseudo-motion trajectories are synthesized via rigid transformations, paired with text corpora and 2D projections derived from raw point clouds. We then propose a Geometric-Motion Duality Network optimized via multimodal contrastive learning, rigid rotation prediction, and motion distribution divergence to produce dense self-supervision. During fine-tuning, we freeze the pretrained backbone and only update a lightweight Spatio-temporal Side Network built with LoRA units. Equipped with a gradient flow masking strategy, PoinTriE simultaneously reduces memory consumption and parameter overhead. Extensive experiments confirm that PoinTriE establishes new state-of-the-art results on action recognition and semantic segmentation tasks.

点云视频自监督轻量微调高效学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。