无需标注伪装物体，实现零样本精准分割。
Towards Real Zero-Shot Camouflaged Object Segmentation without Camouflaged Annotations
- 利用显著性分割特征与掩码图像建模，实现无标注迁移。
- 在CAMO和COD10K上分别达72.9%和71.7%的Fβ^w分数。
- 引入可学习代码本，推理速度达18.1 FPS，适合部署。
伪装物体分割（COS）因标注数据稀缺而面临挑战，像素级标注耗时费力，主要由于物体与背景边界复杂。针对“能否在不依赖任何伪装物体标注的情况下实现有效零样本分割”这一核心问题，本文给出肯定回答，并提出一种鲁棒的零样本COS框架。该框架利用COS固有的局部模式偏差，结合显著性物体分割（SOS）生成的广义语义特征空间，实现高效零样本迁移。框架集成基于掩码图像建模（MIM）的图像编码器（专用于参数高效微调，PEFT）、多模态大语言模型（M-LLM）以及多尺度细粒度对齐（MFA）机制。MIM预训练编码器聚焦提取关键低层特征，M-LLM生成描述嵌入并与视觉线索联合处理，通过MFA实现精准对齐，使模型能准确理解复杂语义上下文。为提升效率，引入可学习代码本以在推理中替代M-LLM，显著降低计算开销。实验验证其卓越性能：在零样本COS任务中，于CAMO数据集上取得72.9%的Fβ^w得分，于COD10K上达71.7%。移除M-LLM后，推理速度可达18.1 FPS，接近传统端到端模型。代码已开源。
原文摘要 · Abstract (English)
Camouflaged Object Segmentation (COS) faces significant challenges due to the scarcity of annotated data, where meticulous pixel-level annotation is both labor-intensive and costly, primarily due to the intricate object-background boundaries. Addressing the core question, "Can COS be effectively achieved in a zero-shot manner without manual annotations for any camouflaged object?" we affirmatively respond and introduce a robust zero-shot COS framework. This framework leverages the inherent local pattern bias of COS and employs a broad semantic feature space derived from salient object segmentation (SOS) for efficient zero-shot transfer. We incorporate an Masked Image Modeling (MIM) based image encoder optimized for Parameter-Efficient Fine-Tuning (PEFT), a Multimodal Large Language Model (M-LLM), and a Multi-scale Fine-grained Alignment (MFA) mechanism. The MIM pre-trained image encoder focuses on capturing essential low-level features, while the M-LLM generates caption embeddings processed alongside these visual cues. These embeddings are precisely aligned using MFA, enabling our framework to accurately interpret and navigate complex semantic contexts. To optimize operational efficiency, we introduce a learnable codebook that represents the M-LLM during inference, significantly reducing computational overhead. Our framework demonstrates its versatility and efficacy through rigorous experimentation, achieving state-of-the-art performance in zero-shot COS with $F_β^w$ scores of 72.9\% on CAMO and 71.7\% on COD10K. By removing the M-LLM during inference, we achieve an inference speed comparable to that of traditional end-to-end models, reaching 18.1 FPS. Code: https://github.com/AVC2-UESTC/ZSCOS-CaMF
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。