提升机器人多模态感知的抗干扰能力,让视觉更可靠。
RLBind: Adversarial-Invariant Cross-Modal Alignment for Unified Robust Embeddings
- 分两阶段训练:先用正常与对抗样本微调视觉分支,再对齐跨模态特征分布。
- 在图像、音频、热成像和视频上测试,鲁棒性显著优于基线模型。
- 适合需要高安全性的机器人导航与操作场景,兼顾准确率与泛化能力。
统一多模态编码器将视觉、音频等传感器融合到共享嵌入空间,是机器人感知与决策的理想基础。然而,机器人部署时视觉分支易受对抗攻击和自然退化影响,鲁棒性成为安全前提。现有防御方法通常仅在CLIP类编码器中对齐干净与对抗特征,忽视更广泛的跨模态一致性,导致增益有限且常损害零样本迁移性能。本文提出RLBind,一种两阶段对抗不变的跨模态对齐框架,用于构建稳健的统一嵌入。第一阶段通过无监督微调清洁-对抗样本对强化视觉编码器;第二阶段利用跨模态对应关系,最小化干净/对抗特征与文本锚点间的差异,同时强制模态间类别级分布对齐。在图像、音频、热成像和视频数据上的大量实验表明,RLBind在干净准确率和范数约束下的对抗鲁棒性方面均持续优于LanguageBind基线和标准微调基准。通过增强抗干扰能力而不牺牲泛化性,RLBind为具身机器人在导航、操作等自主任务中的多传感器感知系统提供了可行的安全路径。
原文摘要 · Abstract (English)
Unified multi-modal encoders that bind vision, audio, and other sensors into a shared embedding space are attractive building blocks for robot perception and decision-making. However, on-robot deployment exposes the vision branch to adversarial and natural corruptions, making robustness a prerequisite for safety. Prior defenses typically align clean and adversarial features within CLIP-style encoders and overlook broader cross-modal correspondence, yielding modest gains and often degrading zero-shot transfer. We introduce RLBind, a two-stage adversarial-invariant cross-modal alignment framework for robust unified embeddings. Stage 1 performs unsupervised fine-tuning on clean-adversarial pairs to harden the visual encoder. Stage 2 leverages cross-modal correspondence by minimizing the discrepancy between clean/adversarial features and a text anchor, while enforcing class-wise distributional alignment across modalities. Extensive experiments on Image, Audio, Thermal, and Video data show that RLBind consistently outperforms the LanguageBind backbone and standard fine-tuning baselines in both clean accuracy and norm-bounded adversarial robustness. By improving resilience without sacrificing generalization, RLBind provides a practical path toward safer multi-sensor perception stacks for embodied robots in navigation, manipulation, and other autonomy settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。