提升图像指代分割的选图准确率，让错误样本更少
Venice-H1: Failure-Aware Query Re-Ranking with Multi-Scale Grid Signatures for Referring Image Segmentation
- 用多尺度网格特征重排候选分割图，识别低质量输出
- 在16组测试中，平均提升1.40 mIoU，失败率下降超半数
- 轻量模块可零样本迁移至医疗图像，无需额外训练
现代指代图像分割系统对每个表达生成多个候选掩码，但仅依赖简单启发式（如argmax得分）选择最终结果。我们发现查询选择是主要失败瓶颈：尽管启发式方法在82%-93%样本上成功，剩余7%-18%的失败却占了大部分误差，存在3%-11%的mIoU提升空间。本文提出Venice-H1，一个轻量、与主干解耦的后处理重排序模块，通过4x4、8x8、16x16网格的多尺度空间特征编码每个候选，并由基于Transformer的失败门控器（ROCAUC 0.78-0.82）判断是否需干预。该模块仅在默认选择可能不佳时介入。在DeRIS-L和DeRIS-B上，分别实现+1.40和+0.89 mIoU的改进，所有16组（数据集，主干）组合的95%置信区间均严格为正，有害切换率低于0.53%。零样本迁移至医疗图像分割（MS-CXR、M3D-RefSeg-2D）分别获得+1.16和+0.51 mIoU，无需微调主干模型。模块增加约1130万参数，延迟低于1毫秒。
原文摘要 · Abstract (English)
Modern Referring Image Segmentation (RIS) systems generate multiple candidate masks per expression but rely on a simple heuristic--typically the argmax detection score--to select the final output. We identify query selection as a failure-case bottleneck: although heuristic selection succeeds on 82-93% of samples, the residual 7-18% of failures dominate the error budget, leaving a best-query selection gap of 3-11% mIoU. We introduce Venice-H1, a lightweight, backbone-decoupled post-hoc re-ranking module that encodes each candidate through multi-scale grid signatures--compact spatial descriptors pooled onto 4x4, 8x8, and 16x16 grids--and feeds them to a Transformer-based re-ranker with a Failure Gate (ROCAUC 0.78-0.82) that intervenes only when the default choice is likely suboptimal. Instantiated on DeRIS-L and DeRIS-B, Venice-H1 achieves delta_fail of +1.40 and +0.89 mIoU with strictly positive 95% CIs on all 16/16 (split, backbone) pairs and harmful-switch rates below 0.53%. Zero-shot transfer to medical referring segmentation (MS-CXR, M3D-RefSeg-2D) yields +1.16 and +0.51 mIoU without RIS-backbone fine-tuning. The module adds approximately 11.3M parameters and under 1 ms latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。