通过分阶段语义量化,提升音乐推荐中多模态兴趣建模精度。
Progressive Semantic Residual Quantization for Multimodal-Joint Interest Modeling in Music Recommendation
- 提出渐进式语义残差量化,保留模态原始语义特征。
- 在多个真实数据集上超越现有方法,线上测试效果显著提升。
- 适合大规模音乐推荐系统,尤其关注跨模态关联的场景。
在音乐推荐系统中,多模态兴趣学习至关重要,可捕捉歌词、乐器、旋律等细粒度偏好。近期基于语义ID的多模态融合方法表现优异,但存在两大缺陷:一是模态内语义退化,残差量化导致离散ID逐渐偏离原始内容语义;二是模态间建模缺失,传统融合策略忽略模态特异性或无法捕捉跨模态相关性。为此,本文提出两阶段多模态推荐框架:第一阶段采用渐进式语义残差量化(PSRQ)生成模态专属与联合语义ID,显式保留前缀语义特征;第二阶段设计多码本交叉注意力(MCCA)网络,同时建模模态特异性兴趣与跨模态关联。在多个真实数据集上的实验表明,该框架优于当前最优基线。该方案已部署于中国头部音乐流媒体平台,线上A/B测试验证了商业指标显著提升,证明其在工业级推荐系统中的实用价值。
原文摘要 · Abstract (English)
In music recommendation systems, multimodal interest learning is pivotal, which allows the model to capture nuanced preferences, including textual elements such as lyrics and various musical attributes such as different instruments and melodies. Recently, methods that incorporate multimodal content features through semantic IDs have achieved promising results. However, existing methods suffer from two critical limitations: 1) intra-modal semantic degradation, where residual-based quantization processes gradually decouple discrete IDs from original content semantics, leading to semantic drift; and 2) inter-modal modeling gaps, where traditional fusion strategies either overlook modal-specific details or fail to capture cross-modal correlations, hindering comprehensive user interest modeling. To address these challenges, we propose a novel multimodal recommendation framework with two stages. In the first stage, our Progressive Semantic Residual Quantization (PSRQ) method generates modal-specific and modal-joint semantic IDs by explicitly preserving the prefix semantic feature. In the second stage, to model multimodal interest of users, a Multi-Codebook Cross-Attention (MCCA) network is designed to enable the model to simultaneously capture modal-specific interests and perceive cross-modal correlations. Extensive experiments on multiple real-world datasets demonstrate that our framework outperforms state-of-the-art baselines. This framework has been deployed on one of China's largest music streaming platforms, and online A/B tests confirm significant improvements in commercial metrics, underscoring its practical value for industrial-scale recommendation systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。