用共享子向量压缩用户物品嵌入,大幅降内存占用
URecJPQ: Memory-efficient Multimodal Recommendation Models through RecJPQ in Large-Scale Scenarios

- 将用户/物品嵌入拆解为共享子向量拼接,减少参数量
- 参数量降低98%-99%,检查点大小减少86%-98%
- 在母婴产品等场景还能提升性能,适合大模型部署
在大规模工业级推荐系统中,用户和物品的ID嵌入导致内存开销巨大,尤其在多模态推荐中,图像、文本等特征进一步加剧资源消耗。本文提出URecJPQ,一种针对大规模与多模态top-k推荐任务的联合产品量化方法。核心思想是将每个用户/物品表示为共享的可学习子嵌入的拼接,而非独立全量嵌入,显著降低可训练参数总量。在电影、母婴、运动用品三个跨领域数据集上的实验表明,在大规模场景下,该方法使检查点大小减少86%至98%,可训练参数量下降98%至99%,平均仅带来8.5%的召回率和16%的NDCG下降,部分场景甚至实现最高85%的性能提升(如母婴产品域)。代码已开源。
原文摘要 · Abstract (English)
Training state-of-the-art recommendation models on large-scale industrial datasets can be a challenging task due to the high number of users and items which are typically represented through ID embeddings. Such embeddings typically require a large amount of memory resources, which are not always available. This problem is further exacerbated in multimodal recommendation, in which multimodal item features generally improve recommendation performance, but require more resources to encode. In this paper, we introduce URecJPQ, a Joint Product Quantization method specifically designed for large-scale and multimodal top-k recommendation tasks, in which the vast number of users and items, combined with the available modalities, further increases the memory demands for the computation. The core idea is to represent each user/item not as a fully learned, unique embedding, but rather as a concatenation of shared learned sub-embeddings, thereby significantly reducing the total number of trainable parameters. Our experiments on three widely-used datasets across different domains (movies, baby and sports products) show that URecJPQ can be effectively applied to multimodal recommendation settings. In large scale scenarios, we observe a substantial reduction in checkpoint sizes and the number of trainable parameters (ranging from 86% to 98%, and 98% to 99%, respectively), with only a marginal decrease in accuracy (8.5% on recall and 16% on NDCG, on average), and, in some cases, even performance improvements (up to 85%), as in the baby products domain. Our codebase is available at https://github.com/giuspillo/urecjpq.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。