轻量级适配器提升SAM对输入退化的鲁棒性
GaRA-SAM: Robustifying Segment Anything Model with Gated-Rank Adaptation
- 在冻结的SAM中间层引入动态调整秩的门控适配器
- 在真实退化数据集ACDC上最高提升21.3%的分割准确率
- 保持原模型泛化能力，适合自动驾驶等高安全场景
提升分割一切模型（SAM）对输入退化的鲁棒性对于自动驾驶和机器人等高风险应用至关重要。本文提出一种轻量级适配器机制——门控秩自适应（GaRA），将其引入冻结的SAM中间层。每个适配器通过学习的门控模块，根据输入动态激活权重矩阵中的秩-1组件，从而实现细粒度且输入感知的鲁棒性增强。该方法兼顾参数效率、精准修复退化输入，并遵循标准训练流程。实验表明，所提模型GaRA-SAM在所有鲁棒分割基准上均显著优于现有方法，尤其在真实退化图像数据集ACDC上，最高达到21.3%的交并比（IoU）提升。
原文摘要 · Abstract (English)
Improving robustness of the Segment Anything Model (SAM) to input degradations is critical for its deployment in high-stakes applications such as autonomous driving and robotics. Our approach to this challenge prioritizes three key aspects: first, parameter efficiency to maintain the inherent generalization capability of SAM; second, fine-grained and input-aware robustification to precisely address the input corruption; and third, adherence to standard training protocols for ease of training. To this end, we propose gated-rank adaptation (GaRA). GaRA introduces lightweight adapters into intermediate layers of the frozen SAM, where each adapter dynamically adjusts the effective rank of its weight matrix based on the input by selectively activating (rank-1) components of the matrix using a learned gating module. This adjustment enables fine-grained and input-aware robustification without compromising the generalization capability of SAM. Our model, GaRA-SAM, significantly outperforms prior work on all robust segmentation benchmarks. In particular, it surpasses the previous best IoU score by up to 21.3\%p on ACDC, a challenging real corrupted image dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。