arXiv:2607.29222cs.CV2026-07

用新基准发现大模型定位强但分割弱,提出无需训练的统一框架

Is It Time for the Renaissance of Salient Object Detection in the Era of MLLMs?

论文配图:Is It Time for the Renaissance of Salient Object Detection in the Era of MLLMs?
图 1 · 摘自论文原文
  • 拆解检测任务为定位与分割,构建新评估基准SaliLLM
  • 大模型定位优于顶尖方法,但分割能力显著落后于标注标准
  • 提出FOCUS框架,利用感知结构实现零样本统一对象组织

多模态大语言模型(MLLMs)的零样本能力正推动显著物体检测(SOD)摆脱特定任务监督。为超越传统基于掩码的评估,我们把SOD分解为定位与分割,并重构数据集以包含短语、边界框和属性,建立用于诊断MLLM显著性感知的基准SaliLLM。SaliLLM揭示了显著的能力错配:MLLM在定位上优于当前最优(SOTA)方法,但在分割上仍明显较弱。进一步分析表明,这一差距主要源于MLLM与标注在前景基数、粒度和范围上的不匹配。基于此诊断,我们重新定义零样本SOD为协议对齐的前景组织,提出首个无需训练的框架FOCUS,该框架采用格式塔启发的协作注意力实现统一SOD。FOCUS将自上而下的协议条件化前景粒度贝叶斯惊喜校准与自下而上的基于自监督特征诱导的实体中心感知流中的证据传播相结合,生成连贯的对象范围作为通用分割器的提示。在13个RGB、RGB-D和RGB-T SOD基准上,FOCUS无需训练即普遍超越SOTA方法,相较于全监督、弱监督和自监督方法,平均绝对误差分别降低11%、34%和48%。研究结果预示着SOD的复兴:从任务特定监督转向零样本前景组织。代码见附录。

原文摘要 · Abstract (English)

The zero-shot capabilities of multimodal large language models (MLLMs) are pushing salient object detection (SOD) beyond task-specific supervision. To disentangle MLLMs beyond conventional mask-based evaluation, we decompose SOD into localization and segmentation, and re-engineer datasets with phrases, boxes, and attributes, establishing a diagnostic benchmark for MLLM saliency perception (SaliLLM). SaliLLM uncovers a striking capability mismatch: MLLMs outperform state-of-the-art (SOTA) methods in localization, yet remain substantially weaker in segmentation. Further analyses attribute this gap primarily to mismatches between MLLMs and annotations over foreground cardinality, granularity, and extent. Motivated by this diagnosis, we recast zero-shot SOD as protocol-aligned Foreground Organization and introduce the first training-free framework that leverages Gestalt-inspired Collaborative attention for Unified SOD (FOCUS). FOCUS couples top-down Bayesian-surprise calibration of protocol-conditioned foreground granularity with bottom-up propagation of MLLMs evidence over entity-centric perceptual manifolds induced by self-supervised features, yielding coherent object extents as prompts for a general segmenter. Across 13 RGB, RGB-D, and RGB-T SOD benchmarks, FOCUS generally surpasses SOTA methods without training, reducing mean absolute error by 11\%, 34\%, and 48\% compared with fully, weakly, and self-supervised methods, respectively. Our findings signal the renaissance of SOD: from task-specific supervision to zero-shot foreground organization. Code is available in the supplementary material.

显著物体检测大模型零样本视觉推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。