用文字引导隐式定位,提升电商图像检索精度。
TIGER-FG: Text-Guided Implicit Fine-Grained Grounding for E-commerce Retrieval

- 用商品文本引导生成聚焦目标的表示,无需物体检测。
- 在两个基准上召回率分别提升6.1和34.4个百分点。
- 适合处理带背景、杂乱布局的电商图像检索场景。
电商图像搜索常以裁剪图作为查询,候选项则由完整商品图和结构化文本构成。这种图文检索存在两种不对称:模态差异——视觉查询需匹配图文组合项;粒度差异——裁剪查询需与含背景和干扰物的完整图像比较。基于检测的方案虽能处理粒度问题,但增加计算成本并易引入误差;而CLIP类编码器虽免于检测,却对背景和无关项敏感。为此,我们提出TIGER-FG,一种文本引导的隐式细粒度定位框架,用于图像到多模态电商检索。该框架利用商品文本作为语义引导,生成聚焦目标的项表示,无需物体检测。我们进一步引入双重蒸馏目标,保持目标区域空间一致性与查询-项相似性结构,从而获得更稳定、更具区分性的多模态表示。此外,我们构建了ECom-RF-IMMR,一个包含10M对训练数据的真实世界基准套件,以及两个评估基准,涵盖标准与杂乱商品布局。TIGER-FG在两个评估基准上的Recall@1分别较最强基线提升6.1和34.4个百分点,仅需85.7M查询侧参数和256维嵌入。在公开电商基准上的结果进一步验证其在噪声环境与一到多检索场景下的泛化能力。代码与数据将公开。
原文摘要 · Abstract (English)
E-commerce image search often takes a cropped image as the query, while each candidate is represented by full item images and structured text. This image-to-multimodal retrieval setting presents two asymmetries: a modality disparity -- a visual query must match image--text items, and a granularity disparity -- a cropped query must be compared with full images containing background context and possible distractors. Detection-based pipelines handle the granularity disparity through explicit localization but incur extra cost and error propagation, whereas CLIP-style encoders avoid detection, but are vulnerable to backgrounds or irrelevant items. To address these limitations, we propose TIGER-FG, a text-guided implicit fine-grained grounding framework for image-to-multimodal e-commerce retrieval. TIGER-FG uses item text as semantic guidance to produce target-focused item representations without object detection for retrieval. We further introduce dual distillation objectives that preserve target-region spatial consistency and query--item similarity structure, yielding more stable and discriminative multimodal representations. In addition, we construct ECom-RF-IMMR, a realistic benchmark suite with a 10M-pair training set and two evaluation benchmarks covering standard and cluttered item layouts. TIGER-FG improves Recall@1 over the strongest baseline by 6.1 and 34.4 percentage points on the two evaluation benchmarks, respectively, with only 85.7M query-side parameters and 256-dim embeddings. Results on public e-commerce benchmarks further demonstrate its generalization to noisy and one-to-many retrieval scenarios. Code and data will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。