arXiv:2505.16065cs.IRcs.CL2025-05被引 2

用大模型生成合成数据,提升社交平台搜索的精准度。

Aug2Search: Enhancing Facebook Marketplace Search with LLM-Generated Synthetic Data Augmentation

  • 用大模型生成多样化、高相关性的查询与商品描述。
  • 1亿条合成数据使搜索效果提升4%,超越纯真实数据训练。
  • 适合想用合成数据增强搜索系统的工程师和研究者。

基于嵌入的检索(EBR)是现代搜索引擎的关键技术,能实现查询与结果间的语义匹配。然而,如Facebook Marketplace这类平台的搜索日志数据缺乏多样性与细节,限制了EBR模型捕捉细微搜索模式的能力。为此,我们提出Aug2Search框架,利用生成式AI(GenAI)模型,以多模态、多任务方式生成高质量合成数据,优化查询-商品相关性。本研究评估了大型语言模型(LLM)在生成合成数据方面的表现,并分析其对EBR模型的影响。实验采用8个Llama模型及1亿条来自Facebook Marketplace的真实日志数据。合成数据生成策略包括:(1)生成查询,(2)增强商品列表,(3)从增强列表中生成查询。在三种数据集上训练EBR模型:采样的互动数据(如“Click”和“Listing Interactions”)、原始数据、合成数据,以及两者的混合。结果表明,Llama模型生成的合成数据具有高度连贯性、相关性与多样性,且幻觉率低。使用1亿条合成数据时,模型在ROC_AUC上最高提升4%,验证了该方法的有效性。此外,相同数据量下,仅用合成数据训练的模型性能常优于仅用真实数据或混合训练的模型。

原文摘要 · Abstract (English)

Embedding-Based Retrieval (EBR) is an important technique in modern search engines, enabling semantic match between search queries and relevant results. However, search logging data on platforms like Facebook Marketplace lacks the diversity and details needed for effective EBR model training, limiting the models' ability to capture nuanced search patterns. To address this challenge, we propose Aug2Search, an EBR-based framework leveraging synthetic data generated by Generative AI (GenAI) models, in a multimodal and multitask approach to optimize query-product relevance. This paper investigates the capabilities of GenAI, particularly Large Language Models (LLMs), in generating high-quality synthetic data, and analyzing its impact on enhancing EBR models. We conducted experiments using eight Llama models and 100 million data points from Facebook Marketplace logs. Our synthetic data generation follows three strategies: (1) generate queries, (2) enhance product listings, and (3) generate queries from enhanced listings. We train EBR models on three different datasets: sampled engagement data or original data ((e.g., "Click" and "Listing Interactions")), synthetic data, and a mixture of both engagement and synthetic data to assess their performance across various training sets. Our findings underscore the robustness of Llama models in producing synthetic queries and listings with high coherence, relevance, and diversity, while maintaining low levels of hallucination. Aug2Search achieves an improvement of up to 4% in ROC_AUC with 100 million synthetic data samples, demonstrating the effectiveness of our approach. Moreover, our experiments reveal that with the same volume of training data, models trained exclusively on synthetic data often outperform those trained on original data only or a mixture of original and synthetic data.

搜索系统合成数据大模型推荐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。