用对比学习+状态空间模型,让用户行为预测更快更准。
User-Centric Modeling of Transactional Sequences with Explainable State Space Models
- 用对比学习生成用户嵌入,再输入Mamba模型作为先验信息。
- 在三个数据集上比纯Mamba快2-3倍,准确率也更高。
- 能解释哪些交易行为最重要,适合需要可解释性的推荐系统。
我们提出一种混合方法,用于用户中心的交易事件序列建模,结合对比表示学习(CoLES)与状态空间模型(SSMs)。尽管对比方法能生成高质量的压缩用户表示,但现有编码器(RNN和Transformer)分别存在梯度消失或二次复杂度问题。Mamba是一种选择性SSM,能高效处理长程依赖,但在个性化用户分析中仍少被探索。我们研究了两种融合策略:(1) 用CoLES嵌入初始化Mamba隐藏状态;(2) 将投影后的CoLES嵌入作为前缀标记插入输入序列。两种方法均在第一步提供有信息量的用户先验。在三个公开数据集(Age:多分类年龄组预测;MBD:多标签产品购买;Taobao:二分类购买预测)上的实验表明,该混合模型在性能上持续优于独立的Mamba和CoLES配合线性分类器,且收敛速度比原始SSM基线快2–3倍。通过离散化步映射和集成梯度进行可解释性分析,发现模型在行为丰富的数据集上具有选择性事件过滤能力,并识别出最关键的交易特征。
原文摘要 · Abstract (English)
We propose a hybrid approach for user-centric modeling of transactional event sequences that combines contrastive representation learning (CoLES) with State Space Models (SSMs). While contrastive methods yield high-quality compressed user representations, existing encoders -- RNNs and Transformers -- suffer from vanishing gradients or quadratic complexity, respectively. Mamba, a selective SSM, efficiently handles long-range dependencies but remains underexplored for personalized user analysis. We investigate two integration strategies: (1)~initializing the Mamba hidden state with a CoLES embedding, and (2)~prepending the projected CoLES embedding as a prefix token to the input sequence. Both approaches supply the model with an informative user prior from the first step. Experiments on three public datasets -- Age (multiclass age-group prediction), MBD (multi-label product acquisition), and Taobao (binary purchase prediction) -- demonstrate consistent improvements over standalone Mamba and CoLES with a linear classifier, with the hybrid models converging 2--3$\times$ faster than the plain SSM baseline. Explainability analysis via discretization-step maps and Integrated Gradients reveals selective event filtering on behavior-rich datasets and identifies the most informative transaction features.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。