arXiv:2608.11190cs.IR2026-08中稿 · RecSys 2026

揭露组推荐评估中的平局陷阱,发现许多所谓进步实为误导。

Are We Really Making Progress in Group Recommendation? Unmasking the Tie-Breaking Illusion

  • 引入平局感知评估协议,精准计算平局随机处理下的指标期望值。
  • 多方法在新评估下性能大幅缩水,排名顺序显著改变。
  • 建议用温度缩放BPR替代额外sigmoid,避免平局虚高问题。

近期组推荐方法在标准基准上报告了显著提升,但这些增益是否真实反映对群体偏好的建模进步仍存疑问。本文揭示:训练时的分数压缩与评估时的确定性平局处理相互作用,引发系统性评估偏差。在BPR目标前增加sigmoid变换会显著提高并列最高分数量,使HR@K和NDCG@K等指标高度依赖平局解决方式。我们在CAMRa2011和Mafengwo数据集上重评了多个代表性方法及其基线,在组推荐与用户推荐场景下采用平局感知协议,计算均匀随机平局处理下的HR@K和NDCG@K精确期望值。结果表明,多数先前报告的改进在新评估下大幅减弱,方法相对排序发生明显变化。进一步分析显示,额外sigmoid可能在优化中起到隐式边界平滑作用,而温度缩放BPR可保留该优势而不引发严重平局膨胀。研究强调平局感知评估对可靠衡量组推荐进展的重要性。代码已开源。

原文摘要 · Abstract (English)

Recent group recommendation methods have reported strong improvements on standard benchmarks, but it remains unclear whether these gains always reflect genuine advances in modeling group preferences. In this paper, we show that several recent methods are affected by a systematic evaluation bias caused by the interaction between training-time score compression and evaluation-time deterministic tie-breaking. Specifically, an additional sigmoid transformation before the BPR objective can greatly increase tied top scores, making top-K metrics such as HR@K and NDCG@K highly sensitive to how ties are resolved. We revisit recent representative methods and their baselines on CAMRa2011 and Mafengwo under both group and user recommendation settings, and evaluate them with a tie-aware protocol that computes the exact expectation of HR@K and NDCG@K under uniform random tie-breaking. Our results show that many previously reported improvements shrink substantially under tie-aware evaluation, and the relative ranking of methods can change markedly. We further show that the additional sigmoid may act as implicit margin smoothing during optimization, and that temperature-scaled BPR can retain much of this benefit without inducing severe tie inflation. Overall, our findings highlight the importance of tie-aware evaluation for establishing reliable progress in group recommendation. The code is available at https://github.com/songduoma/TieAwareGroupRec.

组推荐评估偏差平局处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。