arXiv:2607.26380cs.IR2026-07中稿 · publication at the…

用真实用户数据持续评估学术搜索推荐策略,发现语义相似性最有效。

Continuous Online Evaluation of Recommendation Strategies in Social Science Academic Search

论文配图:Continuous Online Evaluation of Recommendation Strategies in Social Science Academic Search
图 1 · 摘自论文原文
  • 基于语义嵌入的推荐优于词项匹配和点击路径方法。
  • 语义推荐在各类学术资源中表现更优,但效果因信息类型而异。
  • 适合关注学术搜索个性化与实时评估的研究者参考。

在学术搜索引擎中提供相关推荐是一项复杂任务,因其涉及学科多样性、信息类型差异及用户偏好不一。本文通过整合与评估多种推荐系统,解决这一挑战。研究以面向社会科学领域的领域专用搜索引擎 GESIS Search 为平台,该平台提供研究数据、文献、变量及测量工具的访问。为支持对多类推荐策略的持续、实时评估,采用 STELLA 评价框架。实现并对比了包括传统词项相似性、基于 Transformer 嵌入的语义文档相似性,以及基于历史用户会话点击路径的会话推荐算法。结果表明,用户更偏好语义相似性推荐,其表现优于词项相似性和会话推荐方法。然而,不同信息类别中的推荐效果存在差异,反映出信息检索行为随信息类型而异。本研究揭示了将持续评估融入学术搜索推荐系统开发的重要性,有助于提升推荐与用户需求的契合度。

原文摘要 · Abstract (English)

Delivering relevant recommendations in academic search engines is a complex task due to the diversity of subject areas, information types, and user preferences. In this case study, we address these challenges by integrating and evaluating a range of recommendation systems within GESIS Search - a domain-specific search engine for the social sciences that provides researchers with access to research data, publications, variables, and measurement instruments. To support continuous, real-time evaluation of multiple recommendation strategies with actual platform users, we utilize the STELLA evaluation framework. We implement and compare a diverse set of algorithms, including traditional lexical similarity, semantic document similarity by using transformer-based embeddings, and session-based recommendations based on click paths from historical user sessions. Our results show that users prefer recommendations based on semantic similarity, which outperformed term-similarity and session-based methods. However, the performance of recommenders varies across categories within GESIS Search, suggesting that information-seeking behavior differs by information type. Overall, our study provides insights into how continuous evaluation can be incorporated to develop recommendations that better align with the preferences in academic search portals.

学术搜索推荐系统语义相似性持续评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。