arXiv:2605.07125cs.IRcs.AI2026-05

简单图检索法在多个推荐数据集上击败复杂模型,揭示基准测试存在可简化路径。

An Embarrassingly Simple Graph Heuristic Reveals Shortcut-Solvable Benchmarks for Sequential Recommendation

论文配图:An Embarrassingly Simple Graph Heuristic Reveals Shortcut-Solvable Benchmarks for Sequential Recommendation
图 1 · 摘自论文原文
  • 仅基于最近1-2个物品,通过局部图检索和特征相似性排序
  • 在亚马逊运动与音乐数据集上相对提升达38.10%和44.18%
  • 适合评估推荐模型是否真需复杂建模,而非被简单结构误导

序列推荐正转向结合序列模式与语义物品信息的生成式模型。然而这些方法常在少数通用基准上评估,引发关键问题:这些基准是否真的需要现代生成式推荐模型所宣称的先进建模能力?我们通过一个刻意简单的图启发式方法进行基准审计。仅从最近一两个交互物品出发,从几跳的物品转移图中检索候选,并按物品特征相似性排序。该方法无需序列编码器、生成目标或训练,却在Amazon Review Sports和CDs数据集上相对领先基线分别取得38.10%和44.18%的NDCG@10提升。我们发现这种表现反映的是捷径可解性,而非单一启发式的结果。识别出三类使预测更简单的捷径结构:低分支局部转移、特征平滑转移以及对长用户历史依赖有限。即使其中一种强信号也足以使简单局部检索具有竞争力;削弱这些信号后,复杂模型的优势才显现。在14个数据集上,模型排名随数据特性变化显著,但该启发式在10个数据集中仍具竞争力。结果表明,标准基准上的优异表现未必体现高级序列、语义或生成建模能力。我们呼吁在使用基准支持新模型时,应更谨慎地选择数据集并开展数据集级诊断分析。

原文摘要 · Abstract (English)

Sequential recommendation has increasingly shifted toward generative recommenders that combine sequential patterns with semantic item information. Yet these methods are often evaluated on a small set of widely used benchmarks, raising a key question: do these benchmarks actually require the advanced modeling capabilities that modern generative recommenders claim to provide? We conduct a benchmark audit with an intentionally simple graph heuristic. Starting from only the last one or two interacted items, it retrieves candidates from a few-hop item-transition graph and ranks them by item-feature similarity. Despite using no sequence encoder, generative objective, or training, this heuristic matches or outperforms many modern baselines, with relative NDCG@10 improvements of 38.10% and 44.18% over the best competing baseline on Amazon Review Sports and CDs. We show that this behavior reflects shortcut solvability rather than an artifact of one heuristic. We identify three shortcut structures that can make next-item prediction easier than expected: low-branching local transitions, feature-smooth transitions, and limited dependence on long user histories. These shortcuts need not appear together; even one or two strong signals can make simple local retrieval highly competitive, while weakening them makes the benefits of more sophisticated models clearer. Across 14 datasets, model rankings vary substantially with dataset properties, yet the heuristic remains competitive on 10 of them. Our findings suggest that strong performance on standard benchmarks does not always demonstrate advanced sequential, semantic, or generative modeling ability. We call for more careful dataset selection and dataset-level diagnostic analysis when using benchmarks to support claims about new recommendation models.

序列推荐基准审计捷径学习图检索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。