用梯度动作共享降低推荐系统通信开销,提升效率。
Gradients as an Action: Towards Communication-Efficient Federated Recommender Systems via Adaptive Action Sharing
- 将物品嵌入梯度聚类为少量更新动作,替代直接压缩嵌入向量。
- 通信量减少最高达96.88%,在异构环境下保持推荐性能。
- 自适应聚类机制适配不同设备与网络,适合实际部署场景。
联邦推荐系统(FedRecs)面临两大挑战:一是推荐系统中大量物品嵌入导致通信开销巨大;二是异构网络与客户端设备造成训练效率低下。现有压缩方法虽能降低通信量,但因参数误差导致模型性能下降。本文提出通信高效的FedRAS框架,通过动作共享策略将物品嵌入梯度聚类为有限数量的更新动作进行传输,而非直接压缩嵌入矩阵。由于梯度值远小于嵌入向量,约束梯度方向带来的误差更小。同时,引入自适应聚类机制动态调整动作数量,以适应异构环境。在多个公开数据集上的实验表明,FedRAS可使通信负载减少高达96.88%,且在各类异构场景下维持推荐性能。代码已开源:https://github.com/mastlab-T3S/FedRAS。
原文摘要 · Abstract (English)
As a promising privacy-aware collaborative model training paradigm, Federated Learning (FL) is becoming popular in the design of distributed recommender systems. However, Federated Recommender Systems (FedRecs) greatly suffer from two major problems: i) extremely high communication overhead due to massive item embeddings involved in recommendation systems, and ii) intolerably low training efficiency caused by the entanglement of both heterogeneous network environments and client devices. Although existing methods attempt to employ various compression techniques to reduce communication overhead, due to the parameter errors introduced by model compression, they inevitably suffer from model performance degradation. To simultaneously address the above problems, this paper presents a communication-efficient FedRec framework named FedRAS, which adopts an action-sharing strategy to cluster the gradients of item embedding into a specific number of model updating actions for communication rather than directly compressing the item embeddings. In this way, the cloud server can use the limited actions from clients to update all the items. Since gradient values are significantly smaller than item embeddings, constraining the directions of gradients (i.e., the action space) introduces smaller errors compared to compressing the entire item embedding matrix into a reduced space. To accommodate heterogeneous devices and network environments, FedRAS incorporates an adaptive clustering mechanism that dynamically adjusts the number of actions. Comprehensive experiments on well-known datasets demonstrate that FedRAS can reduce the size of communication payloads by up to 96.88%, while not sacrificing recommendation performance within various heterogeneous scenarios. We have open-sourced FedRAS at https://github.com/mastlab-T3S/FedRAS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。