用张量分解压缩推荐数据,让模型训练更快更准。
TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation
- 用张量分解拆解用户-时间-物品关系,生成精炼合成序列。
- 在多个数据集上训练速度提升5倍以上,性能接近全量数据。
- 适合需要快速迭代的推荐系统研发者使用。
在数据驱动型AI时代,推荐系统重心从模型创新转向数据优化。大规模数据虽推动模型性能,却带来高昂训练成本。数据蒸馏成为关键方案,通过压缩数据加速训练并保持性能。然而,处理具有长序列依赖和海量物品的离散用户-物品交互仍具挑战。本文提出TD3:一种基于张量分解的元学习框架下的数据蒸馏方法,用于序列推荐。TD3从原始数据中提炼出一个全表达能力的合成序列摘要。为降低计算复杂度并提取深层潜在模式,采用张量分解将摘要解耦为四个因子:合成用户隐因子、时序动态隐因子、共享物品隐因子及建模三者关联的'关系核心'。此外,设计了双层优化中的代理目标,超越简单性能匹配,实现原数据与合成数据训练模型特征空间对齐。内层采用增强技术使学习器精准拟合合成摘要,从而在外部循环中准确更新。为加速优化并缓解长依赖问题,引入RaT-BPTT进行双层优化。在多个公开数据集上的实验验证了该方法的优越性及跨架构泛化能力。代码已开源。
原文摘要 · Abstract (English)
In the era of data-centric AI, the focus of recommender systems has shifted from model-centric innovations to data-centric approaches. The success of modern AI models is built on large-scale datasets, but this also results in significant training costs. Dataset distillation has emerged as a key solution, condensing large datasets to accelerate model training while preserving model performance. However, condensing discrete and sequentially correlated user-item interactions, particularly with extensive item sets, presents considerable challenges. This paper introduces \textbf{TD3}, a novel \textbf{T}ucker \textbf{D}ecomposition based \textbf{D}ataset \textbf{D}istillation method within a meta-learning framework, designed for sequential recommendation. TD3 distills a fully expressive \emph{synthetic sequence summary} from original data. To efficiently reduce computational complexity and extract refined latent patterns, Tucker decomposition decouples the summary into four factors: \emph{synthetic user latent factor}, \emph{temporal dynamics latent factor}, \emph{shared item latent factor}, and a \emph{relation core} that models their interconnections. Additionally, a surrogate objective in bi-level optimization is proposed to align feature spaces extracted from models trained on both original data and synthetic sequence summary beyond the naïve performance matching approach. In the \emph{inner-loop}, an augmentation technique allows the learner to closely fit the synthetic summary, ensuring an accurate update of it in the \emph{outer-loop}. To accelerate the optimization process and address long dependencies, RaT-BPTT is employed for bi-level optimization. Experiments and analyses on multiple public datasets have confirmed the superiority and cross-architecture generalizability of the proposed designs. Codes are released at https://github.com/USTC-StarTeam/TD3.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。