PAG生成检索的规划信号在查询变化下易崩溃,影响检索效果。
Lost in Decoding? Reproducing and Stress-Testing the Look-Ahead Prior in Generative Retrieval

- 用并行解码生成文档级前瞻信号,指导后续顺序解码。
- 在MS MARCO和TREC-DL上复现了原论文的性能结果。
- 发现拼写微变会导致规划信号失效,需通过查询翻译缓解。
生成式检索(GR)通过自回归生成文档标识符来排序文档。由于许多GR方法依赖于受前缀树约束的束搜索,有限束宽解码下相关前缀易被提前剪枝。计划先行(PAG)通过并行解码计算文档级前瞻先验,引导后续序列解码。我们复现了PAG在推理时的表现,并对其解码行为进行压力测试。使用作者发布的检查点及标识符/前缀树数据,在报告的解码设置下,我们在MS MARCO Dev和TREC-DL 2019/2020上复现了主要有效性结果,并验证了硬件环境下束大小与延迟之间的权衡关系。此外,我们引入了计划漂移诊断,量化意图不变的查询变异如何改变规划器的前n个候选集及最高权重规划标记,并影响引导解码。结果发现,PAG的规划信号对词汇表面形式变化敏感:意图不变的拼写错误可能引发计划坍塌,导致规划候选池大幅偏移,使前瞻奖励失去有效引导,解码退化为较弱的无引导搜索。我们进一步评估了固定索引下的跨语言鲁棒性,使用非英语mMARCO查询检索英文索引,评估无需重新索引的查询侧缓解策略;结果显示查询翻译在该设置下恢复效果最强。总体而言,我们的结果确认了PAG报告的有效性以及规划引导解码的优势,同时表明这些收益依赖于规划信号在真实查询变化和查询-文档不匹配下的稳定性。
原文摘要 · Abstract (English)
Generative retrieval (GR) ranks documents by autoregressively generating document identifiers. Because many GR methods rely on trie-constrained beam search, they are vulnerable to early pruning of relevant prefixes under finite-beam decoding. Planning Ahead in Generative Retrieval (PAG) mitigates this failure mode by using simultaneous decoding to compute a document-level look-ahead prior that guides subsequent sequential decoding. We reproduce PAG at inference time and stress-test its decoding behavior. Using the authors' released checkpoint and identifier/trie artifacts under the reported decoding setup, we reproduce the main effectiveness results on MS MARCO Dev and TREC-DL 2019/2020, and corroborate the reported beam-size-latency trade-off in our hardware setting. Beyond reproduction, we introduce plan drift diagnostics that quantify how intent-preserving query variations alter the planner's top-n candidate set and highest-weight planner tokens, and how these changes affect guided decoding. We find that PAG's planning signal is brittle under lexical surface-form variation: intent-preserving typos can trigger plan collapse, where the planned candidate pool shifts enough that the look-ahead bonus provides little useful guidance, effectively reverting decoding toward weaker unguided search. We further evaluate fixed-index cross-lingual robustness using non-English mMARCO queries against an English index, and assess query-side mitigation strategies that require no re-indexing; query translation provides the strongest recovery in our setting. Overall, our results confirm PAG's reported effectiveness and the benefit of planning-guided decoding under the released inference setup, while showing that these gains depend on the stability of the planning signal under realistic query variation and query-document mismatch.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。