解决多模态推荐中语义令牌冗余和结构错配问题,提升冷启动效果。
Hi-SAM: A Hierarchical Structure-Aware Multi-modal Framework for Large-Scale Recommendation
- 分离共享与模态特异性语义,通过几何对齐与分层量化实现精准令牌化
- 引入层次化记忆锚点注意力机制,压缩序列长度并保留历史上下文
- 在百万级用户平台验证,冷启动场景提升6.55%,优于现有最优模型
多模态推荐因物品具备文本、图像等丰富属性而受到关注。基于语义ID的方法可将信息离散化为紧凑标记,但仍面临两大挑战:(1) 令牌化不优:现有方法(如RQ-VAE)未能解耦跨模态共性与模态特异性细节,导致冗余或崩溃;(2) 架构-数据不匹配:标准Transformer将语义ID视为平坦序列,忽视用户交互、物品及标记的层次结构。将物品扩展为多个标记会加剧序列长度与噪声,使注意力偏向局部细节而非整体语义。本文提出Hi-SAM,一种层次结构感知的多模态框架,包含两项设计:(1) 解耦语义令牌化器(DST):通过几何感知对齐统一模态,并采用粗到细策略量化;共享码本提取共识,模态特异性码本从残差中恢复细微差异,由互信息最小化约束;(2) 层次化记忆锚点变压器(HMAT):通过层次化旋转位置编码(Hierarchical RoPE)分离物品间与物品内子空间,插入锚点标记将物品压缩为紧凑记忆,仅通过压缩摘要访问历史,保留当前物品细节。在真实世界数据集上的实验表明,该方法持续优于最先进基线,尤其在冷启动场景下表现突出。部署于服务数百万用户的大型社交平台,核心线上指标提升6.55%。
原文摘要 · Abstract (English)
Multi-modal recommendation has gained traction as items possess rich attributes like text and images. Semantic ID-based approaches effectively discretize this information into compact tokens. However, two challenges persist: (1) Suboptimal Tokenization: existing methods (e.g., RQ-VAE) lack disentanglement between shared cross-modal semantics and modality-specific details, causing redundancy or collapse; (2) Architecture-Data Mismatch: vanilla Transformers treat semantic IDs as flat streams, ignoring the hierarchy of user interactions, items, and tokens. Expanding items into multiple tokens amplifies length and noise, biasing attention toward local details over holistic semantics. We propose Hi-SAM, a Hierarchical Structure-Aware Multi-modal framework with two designs: (1) Disentangled Semantic Tokenizer (DST): unifies modalities via geometry-aware alignment and quantizes them via a coarse-to-fine strategy. Shared codebooks distill consensus while modality-specific ones recover nuances from residuals, enforced by mutual information minimization; (2) Hierarchical Memory-Anchor Transformer (HMAT): splits positional encoding into inter- and intra-item subspaces via Hierarchical RoPE to restore hierarchy. It inserts Anchor Tokens to condense items into compact memory, retaining details for the current item while accessing history only through compressed summaries. Experiments on real-world datasets show consistent improvements over SOTA baselines, especially in cold-start scenarios. Deployed on a large-scale social platform serving millions of users, Hi-SAM achieved a 6.55% gain in the core online metric.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。