让模型通过多轮对话实现像素级语义分割，理解用户动态意图。
Pixel-Level Reasoning Segmentation via Multi-turn Conversations
- 基于多轮对话追踪用户意图，实现像素级语义推理分割。
- 构建包含2.4万条对话的PRIST数据集，覆盖8300个交互场景。
- 提出MIRAS框架，生成与用户意图对齐的像素级解释，适合交互式视觉任务。
现有视觉感知系统在单轮对话中聚焦区域级分割，依赖复杂明确的查询指令，无法实现像素级推理，难以理解随交互演化的用户意图。本文提出新型任务——像素级推理分割（Pixel-level RS），通过多轮对话追踪用户意图，实现细粒度分割。为建立基准，构建了基于多轮对话的像素级推理分割数据集PRIST，包含24,000条来自8,300个多轮对话场景的标注语句及分割目标。在此基础上，提出MIRAS框架，融合像素级分割与多轮对话理解能力，生成与用户意图一致的像素级解释。在PRIST上的实验表明，该方法在分割性能和大模型推理指标上均优于现有基线。代码与数据已开源。
原文摘要 · Abstract (English)
Existing visual perception systems focus on region-level segmentation in single-turn dialogues, relying on complex and explicit query instructions. Such systems cannot reason at the pixel level and comprehend dynamic user intent that changes over interaction. Our work tackles this issue by introducing a novel task, Pixel-level Reasoning Segmentation (Pixel-level RS) based on multi-turn conversations, tracking evolving user intent via multi-turn interactions for fine-grained segmentation. To establish a benchmark for this novel task, we build a Pixel-level ReasonIng Segmentation Dataset Based on Multi-Turn Conversations (PRIST), comprising 24k utterances from 8.3k multi-turn conversational scenarios with segmentation targets. Building on PRIST, we further propose MIRAS, a Multi-turn Interactive ReAsoning Segmentation framework, integrates pixel-level segmentation with robust multi-turn conversation understanding, generating pixel-grounded explanations aligned with user intent. The PRIST dataset and MIRSA framework fill the gap in pixel-level reasoning segmentation. Experimental results on the PRIST dataset demonstrate that our method outperforms current segmentation-specific baselines in terms of segmentation and LLM-based reasoning metrics. The code and data are available at: https://github.com/ccccai239/PixelRIST.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。