提出按特征频率重分配知识权重，提升推荐系统知识蒸馏效果
Exploring Feature-based Knowledge Distillation for Recommender System: A Frequency Perspective
- 将特征知识分解为不同频率成分，发现传统方法等权重最小化会忽略重要信息
- 设计轻量级FreqD方法，动态增强关键频率知识的权重，性能显著优于现有方法
- 适合关注推荐系统知识蒸馏优化的研究者与工程师
本文从频率视角分析了推荐系统中的基于特征的知识蒸馏。通过将知识定义为特征的不同频率分量，理论上证明常规的特征知识蒸馏等价于对所有知识成分施加相等的损失权重，进而分析该均等分配策略导致重要知识被忽视。针对此问题，我们提出通过重新分配知识权重来强调关键知识。进一步提出了轻量级的FreqD方法，避免逐项计算各知识成分损失带来的计算开销。大量实验表明，FreqD在多个推荐数据集上持续且显著优于当前最优知识蒸馏方法。代码已开源。
原文摘要 · Abstract (English)
In this paper, we analyze the feature-based knowledge distillation for recommendation from the frequency perspective. By defining knowledge as different frequency components of the features, we theoretically demonstrate that regular feature-based knowledge distillation is equivalent to equally minimizing losses on all knowledge and further analyze how this equal loss weight allocation method leads to important knowledge being overlooked. In light of this, we propose to emphasize important knowledge by redistributing knowledge weights. Furthermore, we propose FreqD, a lightweight knowledge reweighting method, to avoid the computational cost of calculating losses on each knowledge. Extensive experiments demonstrate that FreqD consistently and significantly outperforms state-of-the-art knowledge distillation methods for recommender systems. Our code is available at https://github.com/woriazzc/KDs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。