用扩散模型生成精准种子点，实现无需训练的开放词汇分割
Text-to-seed generation: Training-free open-vocabulary seeded semantic segmentation via re-purposing diffusion as text-guided seed generator
- 用稳定扩散模型的文本-区域对应关系生成注意力种子点
- 在多个基准上达到领先性能，零训练成本
- 适合需要快速部署、无标注数据的分割场景
开放词汇语义分割（OVSS）旨在对任意文本查询对应的图像区域进行分割。尽管通用分割模型SAM在分割任务中表现强大，但其独立完成OVSS的能力仍有限。现有方法常依赖其他模型生成粗略掩码后由SAM优化，但当初始掩码不准时该策略不可靠。本文提出将SAM作为由准确对象点（即种子）引导的区域扩展模块，而非依赖不准确的粗略掩码。受经典种子分割启发，我们将OVSS重构为文本引导的种子定位，再进行基于种子的区域扩展。为此，提出Text-to-Seed（T2S）框架——一种无需训练的方法，利用稳定扩散模型的文本-区域对应关系，生成目标类别描述所对应的注意力种子点。这些稀疏种子作为点提示输入SAM，生成完整物体掩码。无需特定任务训练或额外标注，T2S在标准OVSS基准上表现优异，验证了语义定位与种子驱动空间分割结合的有效性。
原文摘要 · Abstract (English)
Open-vocabulary semantic segmentation (OVSS) aims to segment image regions corresponding to arbitrary text queries. Although the Segment Anything Model (SAM) is a powerful foundation model for segmentation, its standalone performance on OVSS remains limited. Existing methods therefore often use SAM to refine coarse masks predicted by other models, but this strategy is unreliable when the initial masks are inaccurate. In this work, we argue that more reliable segmentation can be achieved by exploiting SAM as a region expansion module guided by accurate object points (i.e., seeds) rather than inaccurate coarse masks. Inspired by classical seeded segmentation, we reformulate OVSS as text-guided seed localization followed by seed-based region expansion. To realize this idea, we propose Text-to-Seed (T2S), a training-free framework that leverages the text-to-region correspondence of Stable Diffusion to generate attention-based seed points for target categories described by text. These sparse seeds are then used as point prompts for SAM to produce full object masks. Without task-specific training or additional annotations, T2S achieves strong performance on standard OVSS benchmarks, demonstrating the effectiveness of combining semantic grounding with seed-driven spatial segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。