通用文本嵌入模型无需微调即可在推荐与搜索中表现优异
Do We Really Need Specialization? Evaluating Generalist Text Embeddings for Zero-Shot Recommendation and Search
- 使用大规模语料预训练的通用文本嵌入模型直接应用
- 零样本下优于传统及微调模型，性能提升显著
- 适合追求高效部署的推荐与搜索系统开发者
预训练语言模型（PLMs）广泛用于从商品元数据中提取语义表示，以支持推荐与搜索任务。在序列化推荐中，PLMs通过文本元数据增强基于ID的嵌入；在商品搜索中，则将商品特征与用户意图对齐。近期研究认为需针对特定任务和领域进行微调以提升表示能力。本文挑战这一假设，表明通用文本嵌入模型（GTEs）在未经过专门适配的情况下，仍可实现强大的零样本性能。实验显示，GTEs在序列推荐与商品搜索任务中均优于传统模型及微调模型。我们归因于其更优的表征能力——特征在嵌入空间中分布更均衡。此外，通过主成分分析（PCA）聚焦最具信息量的方向压缩嵌入维度，可有效降低噪声并提升专用模型性能。为保障可复现性，代码库已公开于 https://split.to/gte4ps。
原文摘要 · Abstract (English)
Pre-trained language models (PLMs) are widely used to derive semantic representations from item metadata in recommendation and search. In sequential recommendation, PLMs enhance ID-based embeddings through textual metadata, while in product search, they align item characteristics with user intent. Recent studies suggest task and domain-specific fine-tuning are needed to improve representational power. This paper challenges this assumption, showing that Generalist Text Embedding Models (GTEs), pre-trained on large-scale corpora, can guarantee strong zero-shot performance without specialized adaptation. Our experiments demonstrate that GTEs outperform traditional and fine-tuned models in both sequential recommendation and product search. We attribute this to a superior representational power, as they distribute features more evenly across the embedding space. Finally, we show that compressing embedding dimensions by focusing on the most informative directions (e.g., via PCA) effectively reduces noise and improves the performance of specialized models. To ensure reproducibility, we provide our repository at https://split.to/gte4ps.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。