arXiv:2605.17531cs.CV2026-05

让模型主动问问题,解决模糊描述的图像分割难题

Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarification

论文配图:Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarification
图 1 · 摘自论文原文
  • 通过多轮对话主动澄清用户意图,避免错误猜测
  • 在模糊查询上性能超越现有方法,标准任务也保持领先
  • 适合需要精准交互的智能视觉应用

指代分割旨在根据文本查询分割图像或视频中的目标对象。尽管近年取得显著进展,现有方法始终假设用户输入的查询已足够精确清晰。然而这一假设不切实际:现实中很难要求用户全面审视视觉内容并确保查询唯一无歧义。当遇到此类情况时,现有分割模型常随意猜测用户偏好,导致不良结果。为此,我们提出 IC-Seg——一种新型代理框架,通过多轮对话在分割前主动澄清用户意图。为有效激励该能力,我们进一步引入 Hi-GRPO,一种分层优化策略,在轨迹、对话轮次和步骤层面注入密集且信息丰富的监督信号,促进高效意图澄清,有效减少冗余交互,提升整体对话质量。为评估,我们构建 Ambi-RVOS,一个包含模糊用户查询的指代视频对象分割基准。大量实验表明,IC-Seg不仅在解决模糊查询上大幅优于现有方法,且在标准推理分割基准上仍保持最先进性能。代码与数据将发布于 https://github.com/iSEE-Laboratory/IC-Seg。

原文摘要 · Abstract (English)

Referring segmentation aims to segment the target objects in images or videos based on the textual query. Despite remarkable progress over the past years, existing works always assume that the user-provided queries are already precise and clear. However, this assumption is impractical. In real-world scenarios, it is unrealistic to expect all users to thoroughly review their visual content and carefully ensure their queries are unique and unambiguous. When encountering such cases, existing segmentation models tend to arbitrarily guess the user preferences, often resulting in undesired outcomes. To address this limitation, we propose IC-Seg, a novel agentic framework that proactively clarifies user intent through multi-turn conversation before segmentation. To effectively incentivize this capability, we further introduce Hi-GRPO, a new hierarchical optimization strategy that injects dense and informative supervision signals at the trajectory, turn, and step levels. This strategy encourages efficient intent clarification, effectively eliminating redundant interactions and improving overall dialogue quality. For evaluation, we establish Ambi-RVOS, a referring video object segmentation benchmark with ambiguous user queries. Extensive experiments demonstrate that IC-Seg not only outperforms existing methods by a large margin in resolving ambiguous queries, but also maintains state-of-the-art performance on standard reasoning segmentation benchmarks. Code and data will be released at https://github.com/iSEE-Laboratory/IC-Seg.

指代分割多轮对话视觉理解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。