DIET让推荐系统持续学习时仅用1%-2%数据,性能接近全量训练。
DIET: Learning to Distill Dataset Continually for Recommender Systems
- 构建动态演化数据记忆库,分阶段更新关键样本以匹配长期训练趋势。
- 将原始数据压缩至1-2%,模型迭代成本降低60倍,性能保持一致。
- 适用于大规模推荐系统快速实验,支持跨模型复用,提升研发效率。
现代深度推荐模型在持续学习范式下依赖海量且不断增长的实时行为日志。在大型平台中,为架构对比或迭代而重复使用全部历史数据重新训练模型成本极高,严重拖慢开发进度。为此,本文提出一种面向推荐系统的流式数据蒸馏方法——DIET,构建一个随数据流演化的紧凑蒸馏数据集,保留训练关键信号。与现有静态蒸馏方法不同,DIET将蒸馏数据视为可进化的训练记忆,通过影响感知的内存寻址机制,在双层优化框架下实现有原则的初始化与选择性更新。在大规模推荐基准上的实验表明,DIET可将训练数据压缩至原规模的1-2%,同时保持与全量数据训练一致的性能趋势,模型迭代成本最高降低60倍。此外,所生成的蒸馏数据在不同模型架构间具有良好泛化能力,验证了流式数据蒸馏作为可扩展、可复用的数据基础在推荐系统开发中的潜力。
原文摘要 · Abstract (English)
Modern deep recommender models are trained under a continual learning paradigm, relying on massive and continuously growing streaming behavioral logs. In large-scale platforms, retraining models on full historical data for architecture comparison or iteration is prohibitively expensive, severely slowing down model development. This challenge calls for data-efficient approaches that can faithfully approximate full-data training behavior without repeatedly processing the entire evolving data stream. We formulate this problem as \emph{streaming dataset distillation for recommender systems} and propose \textbf{DIET}, a unified framework that maintains a compact distilled dataset which evolves alongside streaming data while preserving training-critical signals. Unlike existing dataset distillation methods that construct a static distilled set, DIET models distilled data as an evolving training memory and updates it in a stage-wise manner to remain aligned with long-term training dynamics. DIET enables effective continual distillation through principled initialization from influential samples and selective updates guided by influence-aware memory addressing within a bi-level optimization framework. Experiments on large-scale recommendation benchmarks demonstrate that DIET compresses training data to as little as \textbf{1-2\%} of the original size while preserving performance trends consistent with full-data training, reducing model iteration cost by up to \textbf{60$\times$}. Moreover, the distilled datasets produced by DIET generalize well across different model architectures, highlighting streaming dataset distillation as a scalable and reusable data foundation for recommender system development.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。