用稀疏嵌入提升推荐系统效率与可解释性
Efficient Learning of Sparse Representations from Interactions
- 用高维稀疏嵌入替代传统稠密嵌入，兼顾表达力与效率
- 嵌入尺寸最多压缩10倍且精度无损，100倍压缩仅损失2.5%
- 稀疏维度呈现可解释的倒排索引结构，支持页面布局等场景
从交互数据中学习的嵌入所捕捉的行为模式在生产级推荐系统各个阶段至关重要。然而，在初始召回阶段，实践者面临嵌入表达能力与服务组件可扩展性、延迟之间的固有权衡，亟需既紧凑又富有表现力的表示。为此，我们提出一种训练策略，用于学习高维稀疏嵌入层，以替代传统的稠密嵌入层，平衡了效率、表征能力和可解释性。为验证该方法，我们对生产级协同过滤自编码器ELSA进行了改造，在不损失推荐准确率的情况下实现最高10倍的嵌入尺寸缩减，达到100倍缩减时仅损失2.5%。此外，活跃嵌入维度展现出可解释的倒排索引结构，能直接映射模型隐空间，使候选召回模型本身即可集成段级别推荐功能（如2D首页布局）。源代码、附加结果及在线演示见https://github.com/zombak79/compressed_elsa。
原文摘要 · Abstract (English)
Behavioral patterns captured in embeddings learned from interaction data are pivotal across various stages of production recommender systems. However, in the initial retrieval stage, practitioners face an inherent tradeoff between embedding expressiveness and the scalability and latency of serving components, resulting in the need for representations that are both compact and expressive. To address this challenge, we propose a training strategy for learning high-dimensional sparse embedding layers in place of conventional dense ones, balancing efficiency, representational expressiveness, and interpretability. To demonstrate our approach, we modified the production-grade collaborative filtering autoencoder ELSA, achieving up to 10x reduction in embedding size with no loss of recommendation accuracy, and up to 100x reduction with only a 2.5% loss. Moreover, the active embedding dimensions reveal an interpretable inverted-index structure that segments items in a way directly aligned with the model's latent space, thereby enabling integration of segment-level recommendation functionality (e.g., 2D homepage layouts) within the candidate retrieval model itself. Source codes, additional results, as well as a live demo are available at https://github.com/zombak79/compressed_elsa
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。