arXiv:2606.09241cs.IR2026-06被引 1

提升多模态生成检索的编码解码一致性,减少前缀丢失。

Closing the Indexing-Decoding Gap in Multimodal Generative Retrieval via Prefix Retention Optimization

论文配图:Closing the Indexing-Decoding Gap in Multimodal Generative Retrieval via Prefix Retention Optimization
图 1 · 摘自论文原文
  • 通过前缀保留优化,统一编码与解码过程
  • 在9个任务上显著提升目标前缀保留率
  • 适合做多模态检索与生成的算法研究者

多模态生成检索将多模态检索转化为离散标识符生成,无需对齐外部嵌入进行显式相似性搜索。现有方法使用残差量化构建标识符,并通过受限于字典树的束搜索解码。这种组合引入了编码-解码差距:标识符学习目标(如重构和对比损失)在解码过程中未明确强制前缀可区分性。结果是,即使标识符优化良好,仍可能因低秩前缀在束搜索中过早被剪枝。本文理论刻画该差距并推导出前缀存活率与编码与解码中三个可控因素的关系。基于此,提出统一框架PRO,包含三种机制:(i) 前缀排名蒸馏,使用列表级损失使量化前缀排名与预量化嵌入诱导的排名对齐;(ii) 词表调度,从浅层到深层残差量化层级逐步增加码本大小,减少非目标前缀的早期竞争;(iii) 几何得分融合,向量化候选前缀并将查询相似度纳入束搜索评分,进一步缓解编码-解码不匹配。在九个多模态检索任务上的实验表明,PRO显著提升目标标识符前缀保留率,并优于现有基线。

原文摘要 · Abstract (English)

Multimodal generative retrieval formulates multimodal retrieval as discrete identifier generation, eliminating the need for explicit similarity search over external embeddings. Existing approaches construct identifiers via residual quantization and decode them with trie-constrained beam search. This combination introduces an indexing-decoding gap: identifier learning objectives, including reconstruction and contrastive losses, do not explicitly enforce prefix discriminability during decoding. As a result, even well-optimized identifiers can be irreversibly pruned early in beam search due to low-rank prefixes. We theoretically characterize this gap and derive a survival bound that relates prefix retention to three controllable factors in indexing and decoding. Building on this bound, we propose PRO, prefix retention optimization, a unified framework comprising three mechanisms: (i) prefix ranking distillation aligns quantized prefix rankings with those induced by pre-quantization embeddings using a listwise loss; (ii) vocabulary scheduling increases codebook sizes from shallow to deep residual quantization levels to reduce early competition from non-target prefixes; and (iii) geometric score fusion vectorizes each candidate prefix and incorporates its similarity to the query into beam search scoring, further reducing the indexing-decoding mismatch. Experiments on nine multimodal retrieval tasks show that PRO improves retention of target identifier prefixes and outperforms existing multimodal generative retrieval baselines.

多模态检索生成式检索前缀优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。