通过拓扑引导知识蒸馏，让小模型高效处理点云数据
Topology-Guided Knowledge Distillation for Efficient Point Cloud Processing
- 用拓扑感知表示和梯度引导对齐学生模型学习过程
- 模型体积缩小16倍，推理时间减少1.9倍，性能接近教师模型
- 适合自动驾驶等资源受限场景下的点云处理任务
点云处理在自动驾驶和3D目标识别中至关重要，但高性能模型如Point Transformer V3因计算和内存开销大，在资源受限环境下部署困难。本文提出一种新型知识蒸馏框架，利用拓扑感知表示和梯度引导的知识蒸馏，将高容量教师模型的知识有效迁移到轻量级学生模型。该方法在保留点云底层几何结构的同时，通过梯度特征对齐有选择性地指导学生模型学习。在NuScenes、SemanticKITTI和Waymo数据集上的实验表明，所提方法实现约16倍的模型尺寸压缩和近1.9倍的推理时间降低，同时在仅使用激光雷达数据的条件下，于NuScenes上达到当前最佳的分割性能，超越已有知识蒸馏基线。代码已开源：https://github.com/HySonLab/PointDistill
原文摘要 · Abstract (English)
Point cloud processing has gained significant attention due to its critical role in applications such as autonomous driving and 3D object recognition. However, deploying high-performance models like Point Transformer V3 in resource-constrained environments remains challenging due to their high computational and memory demands. This work introduces a novel distillation framework that leverages topology-aware representations and gradient-guided knowledge distillation to effectively transfer knowledge from a high-capacity teacher to a lightweight student model. Our approach captures the underlying geometric structures of point clouds while selectively guiding the student model's learning process through gradient-based feature alignment. Experimental results in the Nuscenes, SemanticKITTI, and Waymo datasets demonstrate that the proposed method achieves competitive performance, with an approximately 16x reduction in model size and a nearly 1.9x decrease in inference time compared to its teacher model. Notably, on NuScenes, our method achieves state-of-the-art performance among knowledge distillation techniques trained solely on LiDAR data, surpassing prior knowledge distillation baselines in segmentation performance. Our implementation is available publicly at: https://github.com/HySonLab/PointDistill
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。