让分割模型按难易程度自动调节推理长度,提升效率与准确率
PixelThink: Towards Efficient Chain-of-Pixel Reasoning
- 结合任务难度与模型不确定性,动态控制推理链长短
- 在新基准上实现更高分割精度与更少推理步骤
- 适合需要高效可解释多模态理解的场景
现有推理分割方法通常通过图像-文本对及掩码标签微调多模态大语言模型,但在分布外场景泛化能力有限,且缺乏显式推理过程。尽管近期研究采用组相对策略优化(GRPO)增强推理能力,但常出现过度推理问题,导致推理链过长且计算开销高,难以控制推理质量。为此,我们提出PixelThink,一种在强化学习框架下集成外部任务难度估计与内部模型不确定性测量的简洁有效方案,使模型能根据场景复杂度和预测置信度自适应压缩推理长度。为支持全面评估,我们构建了包含标注推理参考与难度评分的扩展基准ReasonSeg-Diff,以及一套联合评估分割精度、推理质量与效率的指标体系。实验表明,该方法显著提升了推理效率与整体分割性能。本工作为高效可解释的多模态理解提供了新视角。代码与模型将公开。
原文摘要 · Abstract (English)
Existing reasoning segmentation approaches typically fine-tune multimodal large language models (MLLMs) using image-text pairs and corresponding mask labels. However, they exhibit limited generalization to out-of-distribution scenarios without an explicit reasoning process. Although recent efforts leverage reinforcement learning through group-relative policy optimization (GRPO) to enhance reasoning ability, they often suffer from overthinking - producing uniformly verbose reasoning chains irrespective of task complexity. This results in elevated computational costs and limited control over reasoning quality. To address this problem, we propose PixelThink, a simple yet effective scheme that integrates externally estimated task difficulty and internally measured model uncertainty to regulate reasoning generation within a reinforcement learning paradigm. The model learns to compress reasoning length in accordance with scene complexity and predictive confidence. To support comprehensive evaluation, we introduce ReasonSeg-Diff, an extended benchmark with annotated reasoning references and difficulty scores, along with a suite of metrics designed to assess segmentation accuracy, reasoning quality, and efficiency jointly. Experimental results demonstrate that the proposed approach improves both reasoning efficiency and overall segmentation performance. Our work contributes novel perspectives towards efficient and interpretable multimodal understanding. The code and model will be publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。