用网页检索增强大模型推荐,让推荐更准更实时。
WebRec: Enhancing LLM-based Recommendations with Attention-guided RAG from Web
- 用大模型把推荐任务转为网页搜索查询,提升检索相关性
- 针对网页信息杂乱,设计消息传递头增强长距离相关注意力
- 适合需要实时、个性化推荐的场景,如电商、内容平台
推荐系统在缓解信息过载、提升在线体验方面至关重要。随着大语言模型(LLM)的发展,基于LLM的推荐系统成为主流范式。近期,检索增强生成(RAG)被用于提升LLM的推荐能力,通过引入外部知识库的信息。然而,作为最新信息的重要来源,网络内容在现有RAG推荐中仍被低估。主要挑战来自两方面:一是如何生成契合推荐任务的网页检索查询,因网络搜索与推荐存在知识鸿沟;二是网页内容噪声大,有效信息分散。为此,我们提出WebRec,一种新型基于网页的RAG框架。该框架利用LLM推理能力,将推荐任务转化为适配网页检索的用户偏好查询。此外,针对检索到的嘈杂网页信息,其中有效证据分布零散的问题,设计了消息传递头(MP-Head),通过消息传递机制增强远距离相关片段间的注意力。大量实验验证了所提方法在推荐场景中的有效性。
原文摘要 · Abstract (English)
Recommender systems play a vital role in alleviating information overload and enriching users' online experience. In the era of large language models (LLMs), LLM-based recommender systems have emerged as a prevalent paradigm for advancing personalized recommendations. Recently, retrieval-augmented generation (RAG) has drawn growing interest to facilitate the recommendation capability of LLMs, incorporating useful information retrieved from external knowledge bases. However, as a rich source of up-to-date information, the web remains under-explored by existing RAG-based recommendations. In particular, unique challenges are posed from two perspectives: one is to generate effective queries for web retrieval, considering the inherent knowledge gap between web search and recommendations; another challenge lies in harnessing online websites that contain substantial noisy content. To tackle these limitations, we propose WebRec, a novel web-based RAG framework, which takes advantage of the reasoning capability of LLMs to interpret recommendation tasks into queries of user preferences that cater to web retrieval. Moreover, given noisy web-retrieved information, where relevant pieces of evidence are scattered far apart, an insightful MP-Head is designed to enhance LLM attentions between distant tokens of relevant information via message passing. Extensive experiments have been conducted to demonstrate the effectiveness of our proposed web-based RAG methods in recommendation scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。