arXiv:2501.02837cs.DCcs.AI2025-01KDD被引 6

为不同设备动态构建专属推荐模型,一次前向即可自适应调整。

Forward Once for All: Structural Parameterized Adaptation for Efficient Cloud-coordinated On-device Recommendation

  • 用结构控制器按需组装网络,支持设备定制
  • 仅需一次前向传播完成适配,无须反向传播
  • 避免梯度冲突,适合资源受限的边缘设备

在以云为中心的推荐系统中,用户设备与云端频繁交互会增加带宽压力并带来隐私风险。本地推荐通过在设备端进行重排序来缓解这些问题。现有方法多关注局部自适应参数,却忽视了定制化模型结构的重要性。研究表明,不同数据分布可能需要不同架构以更好拟合。统一模型结构在异构设备上可能导致性能不足或资源浪费。为此,本文提出 Forward-OFA,一种用于动态构建设备专属网络(结构与参数)的新方法。Forward-OFA 使用结构控制器选择每个模块是否组装,训练时通过结构引导映射,将实时行为映射到已组装网络的参数。该映射器内含结构相关参数和并行组件,防止各部分接收异构梯度,从而规避联合优化中的梯度冲突。直接映射使 Forward-OFA 仅需一次前向传播即可实现自适应,快速响应兴趣变化,无需设备端反向传播。在真实数据集上的实验验证了其有效性和高效性。

原文摘要 · Abstract (English)

In cloud-centric recommender system, regular data exchanges between user devices and cloud could potentially elevate bandwidth demands and privacy risks. On-device recommendation emerges as a viable solution by performing reranking locally to alleviate these concerns. Existing methods primarily focus on developing local adaptive parameters, while potentially neglecting the critical role of tailor-made model architecture. Insights from broader research domains suggest that varying data distributions might favor distinct architectures for better fitting. In addition, imposing a uniform model structure across heterogeneous devices may result in risking inefficacy on less capable devices or sub-optimal performance on those with sufficient capabilities. In response to these gaps, our paper introduces Forward-OFA, a novel approach for the dynamic construction of device-specific networks (both structure and parameters). Forward-OFA employs a structure controller to selectively determine whether each block needs to be assembled for a given device. However, during the training of the structure controller, these assembled heterogeneous structures are jointly optimized, where the co-adaption among blocks might encounter gradient conflicts. To mitigate this, Forward-OFA is designed to establish a structure-guided mapping of real-time behaviors to the parameters of assembled networks. Structure-related parameters and parallel components within the mapper prevent each part from receiving heterogeneous gradients from others, thus bypassing the gradient conflicts for coupled optimization. Besides, direct mapping enables Forward-OFA to achieve adaptation through only one forward pass, allowing for swift adaptation to changing interests and eliminating the requirement for on-device backpropagation. Experiments on real-world datasets demonstrate the effectiveness and efficiency of Forward-OFA.

推荐系统边缘计算自适应架构轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。