arXiv:2606.29947cs.IRcs.LG2026-06

LLM推荐器在冷启动场景下表现不佳,根源是召回环节漏掉目标项。

Diagnosing and Mitigating Retrieval Bottlenecks in LLM-Based Cold-Start Recommendation

  • 设计多检索器融合层LHF,提升冷启动物品的召回覆盖率
  • 在内容丰富的领域,召回率最高提升61%,但协同强的领域仅提升7%
  • 揭示当前提示级LLM重排会破坏融合池,限制语义优势发挥

大型语言模型(LLMs)被用于推荐系统中的重排环节,期望其语义理解能力能缓解冷启动与长尾问题。我们通过一个涵盖五个领域的基准测试,明确区分了重排质量与召回覆盖范围。在黄金项保证存在的理想条件下,校准后的LLM重排器在自然流量下未能持续超越强基线;从Qwen3-8B到32B的纵向扩展仅缩小差距,未完全弥合。在更真实的召回环境中,标准单检索器仅将黄金项放入200项候选池的概率为4.6%-22.9%,主因是32%-91%的冷启动目标为无训练交互的新品。我们提出LHF——一种基于验证集训练的多检索器联合池融合层,作为召回端可实现性的基准。LHF是唯一在所有五域中优于单个检索器的方法,在内容丰富领域恢复了17%-61%的理论覆盖空间,但在协同强的领域仅恢复5%-7%。端到端实验显示:非LLM学习型排序能有效利用LHF池,而提示级LLM重排常使其退化。尽管在文本丰富的领域存在语义冷启动优势,但现有“检索-重排”流程难以触及这一潜力。我们公开了基准协议、数据集划分、提示模板、评估工具及可复现资源:数据见https://doi.org/10.5281/zenodo.20991039,代码见https://doi.org/10.5281/zenodo.20993306。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly used as rerankers in recommender systems, with the expectation that semantic understanding will help in cold-start and long-tail regimes. We test this assumption with a five-domain benchmark that explicitly separates reranking quality from retrieval coverage. In a positive-controlled regime where the gold item is guaranteed present, calibrated LLM rerankers fail to consistently outperform strong collaborative and content baselines under natural traffic, and within-family scaling from Qwen3-8B to Qwen3-32B narrows but does not close the gap on most domains. In a retrieval-realistic regime where the gold item is not injected, the bottleneck is more severe: standard single retrievers place the gold item in a 200-item pool only 4.6-22.9% of the time, largely because 32-91% of cold-start targets are brand-new items with no training interactions. We introduce LHF, a validation-trained learned hybrid fusion layer over a multi-retriever union pool, as a retrieval-side realizability baseline. LHF is the only combiner we test that beats every single retriever on all five domains and recovers 17-61% of oracle coverage headroom on content-rich domains, but only 5-7% on collaboratively strong domains. End-to-end experiments reveal the remaining mismatch: learned non-LLM ranking exploits the LHF pool, while prompt-level LLM reranking often degrades it. LLMs exhibit pockets of semantic cold-start advantage, especially in text-rich domains when the item is already present, but this advantage is largely unreachable in current retrieve-then-rerank pipelines. We release the benchmark protocol, splits, prompts, evaluation tooling, and archived reproducibility artifacts: data at https://doi.org/10.5281/zenodo.20991039 and code at https://doi.org/10.5281/zenodo.20993306.

冷启动推荐检索瓶颈LLM重排多检索融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。