用多模态知识蒸馏提升3D点云模型抗攻击能力,训练无开销,效果超越现有方法。
Multimodal Robust Prompt Distillation for 3D Point Cloud Models
- 通过三模态教师(视觉、3D、文本)联合生成轻量提示,指导学生模型学习鲁棒特征。
- 在多种白盒/黑盒攻击下显著优于现有防御方法,且在干净数据上性能更优。
- 训练阶段完成蒸馏,推理时零额外计算,适合部署于资源受限场景。
对抗攻击对基于学习的3D点云模型构成严重威胁,严重影响其在安全敏感应用中的可靠性。现有防御方法普遍存在(1)计算开销高,(2)跨攻击类型泛化能力差的问题。为此,我们提出一种新颖且高效的师生框架——多模态鲁棒提示蒸馏(MRPD),用于提炼鲁棒的3D点云模型。该方法通过将学生点云模型的特征与三个不同教师的鲁棒嵌入对齐来学习轻量级提示:一个处理深度投影的视觉模型、一个高性能3D模型,以及一个文本编码器。为确保可靠的知识迁移,蒸馏过程由置信度门控机制引导,动态平衡各输入模态的贡献。值得注意的是,由于蒸馏仅在训练阶段完成,推理阶段无额外计算开销。大量实验表明,MRPD在多种白盒与黑盒攻击下显著优于现有最先进防御方法,甚至在干净数据上表现更佳。本工作提出了一种高效利用多模态知识的新范式,为构建鲁棒的3D视觉系统提供了实用路径。
原文摘要 · Abstract (English)
Adversarial attacks pose a significant threat to learning-based 3D point cloud models, critically undermining their reliability in security-sensitive applications. Existing defense methods often suffer from (1) high computational overhead and (2) poor generalization ability across diverse attack types. To bridge these gaps, we propose a novel yet efficient teacher-student framework, namely Multimodal Robust Prompt Distillation (MRPD) for distilling robust 3D point cloud model. It learns lightweight prompts by aligning student point cloud model's features with robust embeddings from three distinct teachers: a vision model processing depth projections, a high-performance 3D model, and a text encoder. To ensure a reliable knowledge transfer, this distillation is guided by a confidence-gated mechanism which dynamically balances the contribution of all input modalities. Notably, since the distillation is all during the training stage, there is no additional computational cost at inference. Extensive experiments demonstrate that MRPD substantially outperforms state-of-the-art defense methods against a wide range of white-box and black-box attacks, while even achieving better performance on clean data. Our work presents a new, practical paradigm for building robust 3D vision systems by efficiently harnessing multimodal knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。