解决跨模态生成检索中的信息不对称问题,避免模型强行编造细节。
WIDE: Wildcard Inference with Dynamic Expansion for Cross-Modal Generative Retrieval

- 用自适应熵阈值预设不确定性边界,识别生成盲区。
- 引入通配符动态扩展搜索空间,避免因猜测失败被惩罚。
- 结合语义相似度与生成置信度重排候选,提升检索准确率。
生成式检索通过将表征学习与搜索统一为序列到序列生成任务取得显著进展。然而,将其扩展至跨模态检索时,不同模态间固有的信息不对称性(如简短文本查询与密集视觉候选之间的差距)带来关键挑战。这种结构不匹配导致自回归解码器在标准字典约束束搜索中产生强制幻觉:模型因无法猜出查询中缺失的细粒度信息而受罚,致使无关候选占据高排名。为此,本文提出动态扩展的通配符推理方法(WIDE)。WIDE 首先通过自适应熵阈值(AET)离线校准各层不确定性边界;解码阶段,基于语义盲点检测的通配符解码(AWD)主动发出通配符而非强制确定标识符,动态扩展搜索空间且不产生对数概率惩罚;最后,盲区重排序(BSR)采用混合评分机制,融合离散生成置信度与连续语义相似度评估扩展候选池。在 M-BEIR 基准上的大量实验表明,WIDE 超越现有先进方法,在有效抑制强制幻觉的同时保持紧凑索引结构。
原文摘要 · Abstract (English)
Generative retrieval has demonstrated significant success by unifying representation learning and search into a single sequence-to-sequence generation task. However, extending this paradigm to cross-modal retrieval reveals a critical challenge arising from the inherent information asymmetry across different modalities, such as the gap between concise text queries and dense visual candidates. This structural mismatch causes the autoregressive decoder to suffer from forced hallucination when generating identifiers via standard trie-constrained beam search, where the model is severely penalized for failing to guess fine-grained details absent from the query, allowing irrelevant candidates to hijack top rankings. To address this issue, we propose Wildcard Inference with Dynamic Expansion (WIDE). WIDE employs Adaptive Entropy Thresholding (AET) to calibrate layer-specific uncertainty boundaries offline. During the decoding generation phase, Asymmetry-aware Wildcard Decoding (AWD) detects semantic blind spots and emits wildcards instead of forced deterministic identifiers, dynamically expanding the search space without incurring log-probability penalties. Finally, Blind-Spot Re-ranking (BSR) evaluates the expanded candidate pool using a hybrid scoring mechanism that combines discrete generation confidence with continuous semantic similarity. Extensive experiments on the M-BEIR benchmark demonstrate that WIDE outperforms state-of-the-art generative retrieval methods, effectively suppressing forced hallucination while maintaining compact index structures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。