不依赖标注,用模型自身预测优化边界框精度。
Where Grounding Accuracy Lives on the IoU Curve: Label-Free Inference-Time Boundary Refinement

- 通过高分辨率重检与几何约束,实现无标签的推理时边界修正。
- 在Ref-L4数据集上,[email protected]:0.95提升至76.013%,最高提升3.066个百分点。
- 适用于需精确边界但无法获取标注的视觉定位场景。
视觉-语言模型虽能正确识别指代对象,但输出的边界框往往不够精准。本文研究冻结的直接回答模型是否可利用自身预测,在推理时不依赖目标标注,通过额外一次局部观察来提升精度。提出无标签精度精修(LFPR)方法:将预测的小区域路由至高分辨率处理流程,于上下文裁剪内重新定位表达,仅在固定几何约束下接受候选,并返回坐标中点。在三个证据层级上评估:在31,921个回顾性Ref-L4表达上,mAcc$_{0.5:0.95}$从72.947%提升至76.013%([email protected]从88.531%升至89.725%,[email protected]从55.788%升至61.142%)。在30,969个RefCOCO/+/g表达上,所有数据集的[email protected]、mAcc和平均IoU均提升(合并mAcc提升+0.645,[email protected]提升+0.817),而[email protected]总体不变:路由单独贡献+1.162,但裁剪、约束与融合导致-1.192,抵消了效果,表明无严格IoU影响。在前瞻性的图像无关Flickr30K Entities上,所有指标均提升(mAcc +0.973,[email protected] +1.022),单框变体更显著(mAcc +2.575,[email protected] +3.689)。对两个公开接地专家模型应用相同操作,各项指标均提升(EGM-4B/8B的[email protected]分别提升+1.569/+6.716),延迟约为两倍,体现其与专家训练的组合而非替代关系。移除约束的对照实验在所有指标上均劣于基准,证明约束是关键支撑。结果表明,指代选择与边界精度可部分分离,不同组件影响IoU曲线不同区域,单一阈值无法揭示此行为。
原文摘要 · Abstract (English)
Vision--language models can identify the correct referent while returning an imprecise bounding box. We study whether a frozen direct-answer model can use its own prediction to allocate one additional localized observation without accessing target annotations at inference. Label-free precision refinement (LFPR) routes predicted-small regions to a higher-resolution pass, re-grounds the expression inside a context crop, admits a candidate only under fixed geometric guards, and returns a fixed coordinate-wise midpoint. We report results across three evidence tiers. On 31,921 retrospective Ref-L4 expressions, LFPR raises mAcc$_{0.5:0.95}$ from 72.947\% to 76.013\% ([email protected] 88.531\%$\to$89.725\%, [email protected] 55.788\%$\to$61.142\%). A frozen transfer to 30,969 RefCOCO/RefCOCO+/RefCOCOg expressions improves every dataset at [email protected], mAcc, and mean IoU (pooled mAcc $+0.645$, [email protected] $+0.817$), while [email protected] is unchanged overall: routing alone gains $+1.162$ points there, but crop, guards, and fusion give back $-1.192$, offsetting rather than showing no strict-IoU effect. A prospective, image-disjoint Flickr30K Entities evaluation improves every endpoint (mAcc $+0.973$, [email protected] $+1.022$), more strongly under a single-box variant (mAcc $+2.575$, [email protected] $+3.689$). The same operator applied to two released grounding specialists improves every endpoint ([email protected] $+1.569$/$+6.716$ for EGM-4B/8B) at roughly twice the latency, composing with specialist training rather than replacing it. A genuine unguarded control (guard removed from the same candidates) underperforms the incumbent on every metric, showing the guard is load-bearing. Together, these results show that referent selection and boundary precision are partially separable, with different components moving opposing regions of the IoU curve -- behavior a single threshold cannot reveal.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。