用可执行的空间程序解析遥感图像中的语言指代,无需训练即可精准定位目标。
GeoSelect: Spatial-Program Execution for Training-Free Referring Remote Sensing Image Segmentation

- 将自然语言指代转化为可执行的领域专用空间程序,显式处理位置、顺序等复杂关系。
- 在RRSIS-D和RISBench上分别达到58.86和55.27的mIoU,超过此前最先进方法两倍以上。
- 全程可解释,失败时自动降级为稳健的几何场方案,适合需要透明推理的应用场景。
遥感图像指代分割旨在从航拍图像中定位由自然语言表达指定的对象。现有无训练方法依赖隐式视觉-语言激活或区域-文本相似度,难以控制航空图像中主导的空间、比较和序数关系,无法表达如“最大的船”或“从左数第二个球场”等结构。本文提出GeoSelect,一种无训练流水线,将指代任务重构为类型化空间程序的执行。一个冻结的纯文本语言模型将表达合成小规模领域特定语言,良构性检查器验证程序,确定性执行器运行它。核心抽象是单一评分候选集类型,所有操作符在此下组合:连续几何场将位置与邻近性表示为密集像素级映射,而离散集合与排序操作符引入极值、序数、计数并集和关系构造,这些是场本身无法表达的。由于执行过程显式,每个中间程序、场和排名均可查证;可靠性阶梯将任何失败程序降级为仅含场的特例,确保每条表达均有输出。GeoSelect在RRSIS-D测试集上取得58.86 mIoU,RISBench上取得55.27 mIoU,较此前最优无训练方法在RRSIS-D上提升超两倍,且无需指代监督,在单个GPU上完成。固定候选与分割器属性的对照实验表明性能提升源于显式执行,而非主干网络;假设分解将剩余差距归因于RRSIS-D上的检测召回率不足和RISBench上的选择精度问题;暴露审计确认其对预训练泄露具有鲁棒性。代码将在录用后于项目页发布:https://avalon-s.github.io/GeoSelect/。
原文摘要 · Abstract (English)
Referring remote sensing image segmentation isolates the object named by a natural-language expression in an aerial image. Existing training-free methods resolve the expression through implicit vision-language activations or region-text similarity, which gives weak control over the spatial, comparative, and ordinal relations that dominate aerial referring: they cannot represent constructions such as the largest ship or the second court from the left. We propose GeoSelect, a training-free pipeline that reframes referring as the execution of a typed spatial program. A frozen, text-only language model synthesises the expression into a small domain-specific language, a well-formedness checker accepts the program, and a deterministic executor runs it. The central abstraction is a single scored candidate set type under which every operator composes: continuous geometric fields realise position and proximity as dense pixel-level maps, while discrete set and order operators add the extremum, ordinal, counted-union, and relational constructions that fields alone cannot express. Because execution is explicit, every intermediate program, field, and ranking is inspectable, and a reliability ladder degrades any failing program to a field-only special case, so every expression returns an answer. GeoSelect attains 58.86 mIoU on RRSIS-D test and 55.27 mIoU on RISBench test, more than twice the best prior training-free method on RRSIS-D, with no referring supervision and on a single GPU. A controlled comparison with candidates and segmenter fixed attributes the gain to explicit execution, not the backbone; an oracle decomposition localises the residual gap to detection recall on RRSIS-D and selection on RISBench, and an exposure audit confirms robustness to pretraining leakage. Code will be released upon acceptance at the project page https://avalon-s.github.io/GeoSelect/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。