用强化学习让模型边思考边分割，提升对新提示的适应能力。
LENS: Learning to Segment Anything with Unified Reinforced Reasoning
- 通过统一奖励机制，让模型在分割时生成有逻辑的推理过程。
- 在三个基准上平均cIoU达81.2%，比现有方法最高提升5.6%。
- 适合需要灵活应对新文本提示的视觉理解场景，如机器人交互。
文本提示的图像分割能实现精细的视觉理解，对人机交互和机器人应用至关重要。然而，现有监督微调方法通常忽略测试时的显式思维链（CoT）推理，限制了其对未见提示和领域的泛化能力。为此，我们提出LENS，一个可扩展的强化学习框架，以端到端方式联合优化推理过程与分割。设计跨句子、框、掩码层级的统一强化学习奖励，激励模型生成有信息量的思维链并提升掩码质量。基于公开的30亿参数视觉语言模型Qwen2.5-VL-3B-Instruct，LENS在RefCOCO、RefCOCO+和RefCOCOg基准上平均cIoU达81.2%，优于强基线方法GLaMM最多5.6%。结果表明，由强化学习驱动的思维链推理显著提升了文本提示分割性能，并为更通用的“分割一切”模型提供了可行路径。代码已开源。
原文摘要 · Abstract (English)
Text-prompted image segmentation enables fine-grained visual understanding and is critical for applications such as human-computer interaction and robotics. However, existing supervised fine-tuning methods typically ignore explicit chain-of-thought (CoT) reasoning at test time, which limits their ability to generalize to unseen prompts and domains. To address this issue, we introduce LENS, a scalable reinforcement-learning framework that jointly optimizes the reasoning process and segmentation in an end-to-end manner. We propose unified reinforcement-learning rewards that span sentence-, box-, and segment-level cues, encouraging the model to generate informative CoT rationales while refining mask quality. Using a publicly available 3-billion-parameter vision-language model, i.e., Qwen2.5-VL-3B-Instruct, LENS achieves an average cIoU of 81.2% on the RefCOCO, RefCOCO+, and RefCOCOg benchmarks, outperforming the strong fine-tuned method, i.e., GLaMM, by up to 5.6%. These results demonstrate that RL-driven CoT reasoning significantly enhances text-prompted segmentation and offers a practical path toward more generalizable Segment Anything models (SAM). Code is available at https://github.com/hustvl/LENS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。