arXiv:2605.25730cs.CV2026-05

提出DeCoDrift,解决闭环分割中解码器注意力漂移问题。

DeCoDrift: Stabilizing Decoder Coupling in Closed-Loop Foundation Segmentation

论文配图:DeCoDrift: Stabilizing Decoder Coupling in Closed-Loop Foundation Segmentation
图 1 · 摘自论文原文
  • 通过分析解码器内部信号,发现反馈循环导致注意力错位。
  • 在电子显微图像上,新方法使注意力稳定性和时序一致性提升。
  • 无需重训练,可直接用于现有模型,适合高精度迭代分割场景。

以Segment Anything Model(SAM)为代表的基座分割模型常用于迭代流程,即预测掩码作为下一轮提示。这种闭环动态过程虽常见,但解码器层面的行为仍不明确。我们发现该反馈环会引发一种未被注意的故障模式——解码器耦合漂移,即掩码解码器的交叉注意力逐渐偏离目标物体,导致误差累积。通过内嵌仪器化SAM的掩码解码器,我们提出了无真值的提示-图像耦合度、注意力稳定性与时序一致性度量。在体素级电子显微数据上,这些内部信号显示标准迭代提示相比基于真值锚定的反馈,系统性地降低注意力对齐与时序连贯性。我们将迭代提示形式化为离散时间动力系统,并证明近端锚定可减少反馈环中的误差放大。基于此分析,我们提出DeCoDrift,一种无需训练的推理时稳定框架,通过约束提示更新来保持解码器耦合。大量实验表明,相比标准迭代提示,DeCoDrift持续提升注意力稳定性、时序一致性和分割质量,且无需重新训练或真值监督。更广泛地,我们的结果表明解码器内部动态不仅是诊断工具,更可提供稳定基座分割模型的实际信号。

原文摘要 · Abstract (English)

Foundation segmentation models such as Segment Anything Model (SAM) are now routinely used in iterative pipelines, where each predicted mask is fed back as the next prompt. This practice turns segmentation into a closed-loop dynamical process, yet the decoder-level behavior of these systems remains largely unexamined. We show that this feedback loop can induce a previously overlooked failure mode, decoder coupling drift, in which the mask decoder's cross-attention progressively loses alignment with the target object, causing errors to accumulate across iterations. We study this phenomenon by instrumenting SAM's mask decoder and deriving ground-truth-free measures of prompt-image coupling, attention stability, and temporal consistency. On volumetric electron microscopy data, these decoder-internal signals reveal that standard iterative prompting systematically degrades attention alignment and temporal coherence relative to oracle-anchored feedback. We then formalize iterative prompting as a discrete-time dynamical system and show how proximal anchoring reduces error amplification in the feedback loop. Building on this analysis, we introduce DeCoDrift, a training-free inference-time stabilization framework that constrains prompt updates and preserves decoder coupling across iterations. Across extensive experiments, DeCoDrift consistently improves attention stability, temporal coherence, and segmentation quality over standard iterative prompting, without retraining or ground-truth supervision. More broadly, our results show that decoder-internal dynamics are not merely diagnostic: they provide actionable signals for stabilizing foundation segmentation models in closed-loop use.

分割闭环系统注意力机制稳定性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。